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Preface 



In 2002, Tennessee passed its first Public Charter School legislation. Four 
schools were successful in gaining approval to begin operating in the 2003-2004 
academic year. The second cohort of three charter schools began operation during the 

2004- 2005 academic year, with the third cohort of five schools starting-up during the 

2005- 2006 academic year. According to the charter school law, these charter schools 
were granted “maximum flexibility” to achieve alternative ways for public schools to 
educate school children. Though the flexibility granted to these schools is considered an 
advantage, previous research on charter schools has demonstrated mixed results in 
their success. 

The purpose of the present evaluation study was to examine the progress made 
in program implementation, school climate, and student achievement by the charter 
schools. A “mixed-methods” design, encompassing both qualitative and quantitative 
data, was employed. The questions upon which the evaluation methods are based 
relate to the progress of individual schools and the overall group in implementing 
desired strategies for curriculum, instruction, and organization, and in attaining the goals 
of No Child Left Behind by bringing every child to proficiency in reading and 
mathematics on the TCAP by 2014. It should be noted that student achievement will be 
more fully examined in a supplemental report when student-level data become 
available. 

The work was conducted by the Center for Research in Educational Policy 
(CREP), a state of Tennessee Center of Excellence, located at The University of 
Memphis. 
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Executive Summary 

The purpose of the present evaluation study was to examine the progress made 
in program implementation, school climate, and student achievement by the Tennessee 
charter schools. Six evaluation questions guided the methodology for this study. 
Student achievement results (Question 1) are addressed in a separate report. The 
following evaluation questions (2-6) are addressed in this report. 

2. What is the frequency of usage of various traditional and alternative (student- 
centered) instructional strategies in the charter schools and compared to national 
norms? 

3. What is the school climate at the charter schools and how does the climate 
compare to national norms? 

4. To what degree and levels of quality are the goals and strategies of the charter 
school being implemented? 

5. What are teacher reactions to and experiences in the charter school? What are 
the adequacy and quality of professional development and resources? 

6. What are parent (caregiver) reactions to and experiences with the charter 
school? 

Description of the Charter Schools 

Circles of Success Learning Academy (COSLA) 

COSLA is an urban school located in Memphis, Tennessee. In its third year of 
operation, COSLA enrolled a total of 98 students in grades K-4. All were African 
American, and most (87%) were eligible for free or reduced price lunch. As a literacy- 
focused school using the Success for All reading program, COSLA emphasizes the 
integration of reading and writing activities across all curricular areas. 

Memphis Academy of Health Sciences (MAHS) 

MAHS is located in a high-poverty area north of downtown Memphis, Tennessee. 
The school served 300 6-8 grade students in its third year of operation. African 
American students represented 100% of the school’s population; 75% of the students 
received free or reduced price lunches. The curriculum is a standards-based, 
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interdisciplinary program that incorporates interdisciplinary projects and experiential 
learning centered on a health science theme. 

Memphis Academy of Science and Engineering (MASE) 

MASE is housed in an office building in downtown Memphis. During the 2005- 
2006 school year, there were 376 seventh, eighth, and ninth grade students. The 
students were predominantly African American (98%), and 68% qualified for free or 
reduced-price lunch. The program includes longer days, Saturday school, high 
technology use, continuous monitoring of student progress, and an intense focus on 
core curricula integrated across subjects. 

Smithson-Craighead Academy (SC A) 

This urban charter school, located in the city of Nashville, Tennessee, is in its 
third year of operation. It served 188 students in kindergarten though fourth grade. The 
students were predominantly African American (99%), and nearly all (99%) were eligible 
for free or reduced price lunch. The mission of the school is to meet the academic and 
social needs of at-risk children through mastery of basic academic skills, student self 
control, obedience, and diligence. 

City University School of Liberal Arts (CityU) 

Cityll is located in a Baptist church in an urban area of Memphis, Tennessee. 
The school enrolled 147 ninth and tenth grade students in its second year of operation. 
Just over 91% of the students were African American, and 39% were eligible for free or 
reduced price lunch. The curriculum developed and employed at CityU has a liberal 
arts core and is aligned with College Board standards to build a strong college-oriented 
focus. 

Star Academy 

Star Academy is located in the outskirts of Memphis, Tennessee. In its second 
year of operation the school served 154 students in kindergarten through third grade. 
The student population was predominantly African American, with approximately 95% of 
the students qualifying for free or reduced price lunch. Reading, language arts, and 
mathematic were emphasized during the academic year. 
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Yo! Academy 

Yo! Academy for the Visual and Performing Arts is located in southwest Memphis 
in an industrial warehouse site. During the second year of operation, the school served 
163 students in the ninth through twelfth grades. All students were African American 
and approximately 80% were eligible for free or reduced price meals. The mission of 
the school is to provide at-risk students with a safe environment to pursue rigorous 
academic preparation while incorporating the visual and performing arts. 

KIPP Academy Nashville 

KIPP Academy is located in a high-poverty section of east Nashville in a 
previously abandoned building owned by the Metropolitan Nashville Public School 
System. During its first year of operation, the school served 57 fifth grade students. 
Most (93%) of these students were African American and 88% were eligible for free or 
reduced price lunches. The Knowledge is Power Program (KIPP) school is 
distinguished by several extensions to the typical school calendar: extended school 
hours; Saturday school held two weeks out of each month; and an extended school 
year with a four week summer program. 

Memphis Business Academy (MBA) 

MBA is located in the business district in the heart of downtown Memphis. In its 
first year of operation, the school served 62 sixth grade students. Almost all (99%) were 
African American and 72% were eligible for free or reduced price lunches. The school 
incorporates business education, entrepreneurship, and financial responsibilities into 
the core curriculum. 

Promise Academy 

Promise Academy is located in Memphis and currently housed in St. Paul’s 
Episcopal Church. The 2005-2006 school year was the school’s first in operation, during 
which only kindergarten classes were conducted. Of the 53 students served for during 
the first year, 98% were African American and 91% were eligible for free or reduced 
price lunches. The school’s mission is as follows: “to prepare children to excel in the 
nation's most rigorous high schools. Our central and only work is to teach and inspire 
the mind, body and spirit of our children so that they can succeed in any academic or 
cultural setting.” 
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Soulsville Charter School 

The Soulsville Charter School is located in a lower socio-economic urban area in 
Memphis, on the same property as the historic Stax Museum of American Soul Music. 
In the school’s first year of operation, 56 sixth grade students were served. All students 
were African American and 91% qualified for free or reduced price lunches. The school 
incorporates a concentration on orchestra and musical skills development into the core 
curriculum developed by Memphis City Schools. The school’s plan includes extended 
school days and half-day Saturday classes. 

Southern Avenue Charter School for Academic Excellence 

Southern Avenue Charter School for Academic Excellence is located in Memphis 
and housed at Greenwood Christian Methodist Episcopal Church, near the University of 
Memphis campus. During its first year of operation, 65 kindergarten and first grade 
students were served. In addition to the basic curriculum, “kaleidoscope” activities such 
as art, music, ballet and tae kwon do are offered. The school calendar and school days 
are extended at Southern Avenue, with “kaleidoscope” classes beginning at 3:00 in the 
afternoon. The school supplements the students’ education with reading and math 
tutorials during an after school program, which runs from 4:00 pm to 6:00 pm. 

Method 

A mixed methods design, encompassing both qualitative and quantitative data, 
was employed. The instruments used in the data collection were: (1) School 
Observation Measure (SOM®), (2) Rubric for Student Centered Activities (RSCA®), (3) 
School Climate Inventory (SCI®), (4) principal interview, (5) teacher focus group, (6) 
student focus group, (7) Charter School Teacher Questionnaire, (8) Charter School 
Parent Questionnaire, and (9) Program Implementation Benchmarking. 

A “site researcher” from CREP was assigned major data collection responsibility 
for each charter school. This individual and support research staff visited the assigned 
school several times during the year to conduct the observation visits, administer the 
questionnaires, and conduct the interview and focus groups. In addition, the site 
researcher worked with the school leadership team to develop “implementation 
benchmarks” describing beginning, intermediate, and full implementation phases and 
associated evidence indicators. At the end of the year, the site researcher met again 
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with the leadership team to determine and identify the phase that had been achieved for 
each benchmark. Data from all instruments were then used to prepare individual school 
formative evaluation reports indicating status and progress during the year. 

Results 

Question 2 1 . What is the frequency of usage of various traditional and alternative 
(student-centered) instructional strategies in the charter schools and compared 
to national norms? 

Across schools and cohorts, teacher-centered instruction remained the dominant 
orientation. The rates of teacher-centered instruction were comparable to national 
norms in many cases, but still reflect limited success by the schools to implement the 
more innovative pedagogy described in benchmarks and instructional plans. The 
anticipated trend that charter schools might move from more traditional to more student- 
centered strategies was not completely realized because many schools were rated as 
moderate or weak in terms of utilizing student-centered strategies. The pattern of 
findings was similar for the category of higher-order instruction, but national norms also 
do not reflect extensive levels of higher-order instruction. Technology usage was the 
least prevalent category of instruction across charter schools. In some schools no 
technology use was observed, whereas in others technology was occasionally used to 
support learning. In many cases the levels of technology use did not compare favorably 
with the normative data. 

Question 3. What is the school climate at the charter schools and how does the 
climate compare to national norms? 

Overall, school climate remains a definite strength of these charter schools. In 
some schools, the findings showed increases from already high averages when 
compared to previous years. In 1 st cohort schools, the decline in scores observed in the 
previous year appeared to rebound. On all dimensions, the SCI ratings obtained for the 
charter schools were higher than national norms. Still, as true in the national sample, 
scores on Order tended to be lower than those on other dimensions. Nevertheless, less 
favorable evaluation of school climate was the exception across charter schools. 



1 Question 1 is addressed in a separate report. 
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Question 4. To what degree and levels of quality are the goals and strategies of 
the charter school being implemented? 

Not surprisingly, the most advanced levels of implementation were observed 
among first cohort schools in their third year of operation. First cohort schools were 
either in intermediate or final phases of implementation across categories. Ratings for 
the implementation of goals and strategies tended to be moderate or intermediate for 
second cohort schools. Low to intermediate levels of implementation were apparent 
among third cohort schools in their initial year of operation. Across schools, the 
strongest levels of implementation tended to be for benchmarks targeting support and 
organization; whereas, lower levels of implementation were more apparent in the areas 
of curriculum, instruction, and evaluation. 

Question 5. What are teacher reactions to and experiences in the charter school? 
What are the adequacy and quality of professional development and resources? 

Overall, teachers’ reactions to and experiences in the charter schools tended to 
be very positive. Responses related to support of the educational program, 
understanding of the mission, and the likelihood for student success were particularly 
favorable. All charter schools were considered to be strong in this category. Most 
charter schools were also rated as strong in the area of professional development. 
Teacher perceptions of the availability and adequacy of resources were more varied 
with most schools rated as moderate. Although there were differences by school in the 
type of resources that respondents endorsed as sufficient or adequate, having enough 
common planning time and/or faculty within the school more commonly emerged as 
areas in need of improvement. With regard to perceptions of support or assistance from 
educational agencies, teachers’ ratings and comments were less favorable. They 
consistently reported a lack of effective assistance provided by state and local 
educational agencies. Regarding longitudinal trends, improvement in teacher 
responses were noted in several schools, and patterns related to cohorts were not 
detected. 
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Question 6. What are parent (caregiver) reactions to and experiences with the 
charter school? 

All 12 charter schools were rated as strong with respect to parent satisfaction. 
These findings were similar to those obtained in previous years. Clearly, the parents 
had primarily favorable reactions to these schools. The levels of parental involvement 
were more varied across schools, with most considered to be strong or moderate in this 
category. There was a pattern of more moderate to low levels of involvement in third 
cohort schools, which seems reasonable for schools just launching their programs. A 
pattern of findings by school level was also detected. Three of the four schools judged 
to have high levels of parent involvement served early elementary grade levels. With 
one exception, the charter schools rated as low in terms of parent involvement were 
third cohort secondary schools. 

Recommendations 

Based on the overall findings, the following recommendations apply to the 
charter schools as a group and not necessarily to individual schools. The 
recommendations specific to each school can be found in the individual reports 
provided to each school. Due to a consistent pattern of findings across years, the 
recommendations are similar to those presented in previous reports. 

First, charter schools might adopt a wider array of instructional orientations or 
strategies shown to promote student achievement. Many of the research-based 
practices and the schools’ own benchmarks call for the implementation of more student- 
centered strategies that entail higher-order thinking and technology integration. 
Traditional direct instruction can be an effective orientation, but a better balance among 
a variety of effective pedagogical practices might be achieved. 

Second, benchmark documents need to be modified to better align with objective 
indicators and available data. Planning, evaluation, and ultimately improvement can be 
facilitated by a sound, ambitious, and long-range benchmark document. An honest, 
accurate appraisal of the extent to which benchmarks have been successfully 
implemented will enhance school improvement efforts. 

Third, efforts to continue developing supportive, collaborative relationship with 
the school district and external partners should continue. Strategies include having 
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more open channels of communication and holding meetings to clarify policies, 
procedures, and available resources. Furthermore, representatives of charter schools 
should continue to seek the support and assistance of various external partners. 
However, we recognize that the schools’ potential progress in establishing and 
furthering such relationships is directly dependent on the degree of involvement desired 
and investment made by the respective external organizations. 

A fourth recommendation is to continue efforts to increase active parent 
involvement or participation. The definition and types of parental involvement might be 
reconsidered. In some school improvement plans, involvement translates into support 
for and understanding of the mission and educational program or maintaining regular 
contact. In others, it specifies more active participation in parent organizations, school 
activities, and the provision of academic support for students. Formal contracts or 
agreements that specify activities and extent of participation might be implemented. 

The final recommendation is to prioritize, coordinate, and secure more resources. 
The selection of targeted resources would be guided by the school improvement plan. 
Admittedly, this recommendation is the most difficult to achieve. Additional resources 
may be garnered by working toward achieving a collaborative, supportive relationship 
with the school district, state educational agency, and external partners. 



113 




Third-Year Evaluation of Tennessee Charter Schools (2005 - 2006) 

This report presents results from the evaluation by the Center for Research in 
Educational Policy of the charter schools operating in the State of Tennessee. The first 
cohort of four schools began operation in the 2003-2004 academic year and have 
completed their third full year of operation. The second cohort of three schools 
completed their second academic year, and the third cohort of five schools completed 
their initial year of operation during the 2005-2006 academic year. By way of historical 
background, the passage of the first charter school law and establishment of the first 
charter school occurred in Minnesota in 1991. As of September 2006, approximately 
4,000 charter schools were operating in the United States (Center for Education 
Reform, 2006). Additionally, under the No Child Left Behind legislation, children who 
attend schools identified as needing improvement have the opportunity to enroll in 
charter schools located within their district (U.S. Department of Education, 2001 ). 

In 2002, the state of Tennessee passed its first public charter school legislation. 
According to this law, charter schools were intended to receive “maximum flexibility” to 
achieve alternative ways for public schools to educate school children. Though the 
flexibility granted to these schools is considered an advantage, previous research on 
charter schools has demonstrated mixed results in their success. 

Although this legislation marks an important precedent in Tennessee policy, it is 
equally important to determine and document the effectiveness of charter schools 
relative to traditional schools in promoting effective educational practices and raising 
student achievement. 
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The purpose of the present evaluation study was to examine the progress made 
in program implementation, school climate, and student achievement by the Tennessee 
charter schools. A “mixed-methods” design, encompassing both qualitative and 
quantitative data, was employed, as will be described in the following sections. The 
questions upon which the evaluation methods are based relate to the progress of 
individual schools and the overall group in implementing desired strategies for 
curriculum, instruction, and organization, and in attaining the goals of No Child Left 
Behind by bringing every child to proficiency in reading and mathematics by 2014. 

In the following section, evaluation questions and the assumptions from which 
they were derived are presented. Table 2 shows each question with associated data 
collection tools, which in turn will be described in more detail in the Methodology 
section. 

Charter School Assumptions and Associated Evaluation Questions 

1. Assumption : The major goal of school reform (i.e., specifically the creation of a 
charter school) is to raise student achievement and improve educational 
outcomes (attendance, socialization, etc.). 

■ Question: What are the immediate and long-term impacts of the charter 
school implementation on student achievement, attendance, and attrition/ 
dropout rates? 2 

2. Assumption: High student achievement is fostered by effective teaching. 

■ Questions: (A) What is the frequency of usage of various traditional and 
alternative (student-centered) instructional strategies in the charter schools?; 
(B) What is the frequency of usage of selected instructional strategies 
associated with best practices?; (C) How do charter instructional practices 
compare to those reflected in national norms? 



2 At the time this report is being prepared, student-level data from the Spring 2006 Tennessee 
Comprehensive Assessment Program/Achievement Test (TCAP/AT) are not yet available. 
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3. Assumption: Positive educational outcomes at schools are fostered by positive 
school climate. 

■ Questions: (A) What is the level of school climate at each charter school?; 
(B) Does school climate improve over time?; (C) How do charter school 
climate outcomes compare to those reflected in national norms? 

4. Assumption: To impact schools positively, educational reforms must be properly 
implemented. 

■ Question: To what degree and levels of quality are the identified goals and 
strategies of each charter school being implemented? 

5. Assumption: To be implemented effectively, educational reforms must have 
teacher support, sufficient professional development, and adequate resources. 

■ Questions: (A) What are teacher reactions to and experiences in TN charter 
schools?; (B) What are the adequacy and quality of professional development 
and needed resources? 

6. Assumption: The effectiveness and sustainability of educational reform and 
intervention programs are enhanced when students’ families are supportive and 
involved. 

■ Questions: (A) What are parent (caregiver) reactions to and experiences with 
TN charter schools?; (B) To what degree and in what ways are caregivers 
involved with each charter school? 



Methodology 

Description of the Charter Schools 

As noted, four schools comprise the first cohort, three comprise the second 
cohort, and five schools were included in the third cohort. Table 1 presents an overview 
of the schools by cohort. A more detailed, narrative description of each school follows 
the table. 
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Table 1 

Overview of Schools 



School 


Cohort 


Level 


Grades 

(05-06) 


Enrollment 

(05-06) 


Location 


Circles of Success Learning Academy (COSLA) 


1 


Elementary 


K-4 


98 


Memphis 


Memphis Academy of Health Sciences (MAHS) 


1 


Middle 


6-8 


300 


Memphis 


Memphis Academy of Science & Engineering 
(MASE) 


1 


Secondary 


7-9 


376 


Memphis 


Smithson-Craighead Academy (SCA) 


1 


Elementary 


K-4 


188 


Nashville 


City University of School of Liberal Arts (CityU) 


2 


Secondary 


9-10 


147 


Memphis 


Star Academy 


2 


Elementary 


K-3 


154 


Memphis 


Yo! Academy for the Visual and Performing Arts 


2 


Secondary 


9-12 


163 


Memphis 


KIPP Academy Nashville 


3 


Middle 


5 


57 


Nashville 


Memphis Business Academy (MBA) 


3 


Middle 


6 


62 


Memphis 


Promise Academy 


3 


Elementary 


K 


53 


Memphis 


Soulsville Charter School 


3 


Middle 


6 


56 


Memphis 


Southern Avenue Charter School for Academic 
Excellence 


3 


Elementary 


K-1 


65 


Memphis 



Circles of Success Learning Academy (COSLA). COSLA is an urban school 
located in Memphis, sharing space with a church and a day care facility. In its third year 
of operation, the 2005-2006 school year, COSLA enrolled a total of 98 students in 
kindergarten through fourth grade. All were African American, and most (87%) were 
eligible for free or reduced price lunch. COSLA’s staff consisted of five full time teachers 
with a student to teacher ratio of 20:1. Other staff members included a new principal, a 
new administrative assistant, a part-time special education teacher and two teacher 
assistants. As a literacy-focused school using the Success for All reading program, 
COSLA emphasizes the integration of reading and writing activities across all curricular 
areas. Organizational structures are designed to facilitate high levels of learning and 
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therefore include the systematic monitoring of grade level planning, classroom teaching 
practices, and student progress. The school also has a fine arts program during the 
extended day. 

Memphis Academy of Health Sciences (MAHS). MAHS has been housed in the 
upper floor of Caldwell Elementary School, located in a high-poverty area north of 
downtown Memphis. The school served 300 sixth, seventh and eighth grade students 
during the 2005-2006 school year, its third year of operation. The curriculum is a 
standards-based, interdisciplinary program that incorporates interdisciplinary projects 
and experiential learning centered on a health science theme. MAHS has an active 
parent group called “MAHS 100 Parent Council” that meets twice a month. This group 
supports strong parental participation in the school. They have created a parent contract 
that encourages a minimum of 20 hours of volunteer work at school during the school 
year. All stakeholders in the school community are looking forward to moving to another 
facility for the 2006-2007 year at Heritage Baptist Church in Nutbush, within the Berclair 
community in northeast Memphis. 

Memphis Academy of Science and Engineering (MASE). MASE is housed in a 
former medical office building within the heart of the mid-town Memphis medical 
community. During the 2005-2006 school year, the school’s third year in operation, 
there were 376 seventh, eighth and ninth grade students. The students were 
predominantly African American (98%), and 67.8% qualified for free or reduced-price 
lunch. The school’s staff consisted of twenty full-time teachers and three part-time 
teachers. The student to full-time teacher ratio was 18:1. The school’s program 
includes longer days, Saturday school, high technology use, continuous monitoring of 
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student progress and an intense focus on core curricula integrated across subjects. 
Role modeling, labs/activity periods and project-based Saturday programs are also 
employed at the school. The school employs a number of strategies not found in the 
traditional school, such as flexible class times, student access to laptops during the 
school day for research and completing assignments, and textbooks on CD. 

Smithson-Craighead Academy (SCA). Smithson-Craighead Academy is an 
urban Nashville school in its third year of operation. The school is housed in an older 
building which previously served as an elementary school within the Nashville Metro 
District School System. It served 188 students in kindergarten through fourth grade 
during the 2005-2006 school year. The students were predominantly African American 
(99%). 99% of students were eligible for free or reduced price lunch. The school 
maintained a staff of eleven full-time teachers, and one part-time teacher. The student 
to full-time teacher ratio was 18:1. The mission of the school is to meet the academic 
and social needs of at-risk children through mastery of basic academic skills, student 
self control, obedience, and diligence. Smithson follows the same calendar as the 
district and continues to offer a summer school session. The school also offers several 
popular after-school programs and a variety of extra-curricular activities. 

City University School of Liberal Arts (CityU). CityLl is located in a Baptist church 
in an urban area of Memphis. The school enrolled 147 ninth and tenth grade students in 
its second year of operation, the 2005-2006 school year. Most (91%) of the students 
were African American, and 39% were eligible for free or reduced price lunch. The 
school’s faculty and staff consisted of six full-time teachers, ten part-time teachers, and 
one paraprofessional. The student to teacher ratio was 15:1. The curriculum developed 
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and employed at CityU has a liberal arts core and is aligned with College Board 
standards to build a strong college-oriented focus. This theme is continued throughout 
City University’s operations, from teachers being referred to as “professors” and 
students as “scholars.” The educators at City University indicate they want their 
students to be prepared for college and are providing them with an opportunity to learn 
in an atmosphere they would not experience in most high schools. 

Star Academy. Star Academy is located in northeast Memphis. The school is 
housed in a former preschool building on the grounds of a Baptist church. In its second 
year of operation, the 2005-2006 school year, the school served 154 students in 
kindergarten through third grade. The student population was predominantly African 
American (99%), with approximately 95% qualifying for free or reduced price lunch. The 
school maintained a staff of seven full-time teachers, two part-time teachers, and three 
paraprofessionals. The student to full-time teacher ratio was 22:1 . The primary focus of 
Star Academy is “back to basics,” which is an emphasis on literacy and mathematical 
competencies. Additional programs include character education, exploratory lessons in 
visual and creative arts (varies according to grade level), music, technology (third 
grade), and physical education. STAR utilizes a unique blend of student/teacher 
interactions along with formal teaching strategies, with the goal of meeting the 
instructional needs of all student learners. 

Yo! Academy for the Visual and Performing Arts. Yo! Academy of Visual and 
Performing Arts is located in southwest Memphis in an industrial warehouse site, a 
facility used as part of the Yo! Memphis youth opportunity program, originally funded by 
the U. S. Department of Labor. During its second year of operation, the 2005-2006 
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school year, the school served 163 students in the ninth through twelfth grades. All 
students were African American and 80% were eligible for free or reduced price meals. 
The school maintained a staff of eight full-time teachers, two part-time teachers, and 
two paraprofessionals. The student to full-time teacher ratio was 16:1. The mission of 
the school is to provide at-risk students with a safe environment to pursue rigorous 
academic preparation while incorporating the visual and performing arts. The school’s 
program includes longer days with performing arts classes scheduled in the latter part of 
the day. They, and the performances that result, constitute an important part of the 
identity of Yo! Academy. 

KIPP Academy Nashville. KIPP Academy is located in a high-poverty section of 
east Nashville in a previously abandoned building owned by the Metropolitan Nashville 
Public School System. During its first year of operation, the 2005-2006 school year, the 
school served 57 fifth grade students. Most (93%) of these students were African 
American and 88% were eligible for free or reduced price lunch. The school maintained 
a staff of two full-time teachers, five part-time teachers and one paraprofessional. The 
principal also taught for part of the day. The student to teacher ratio as reported by 
KIPP Academy was 15:1 . The school was chartered as a Knowledge is Power Program 
(KIPP) and lists its characteristics and unique school features as “Work Hard,” “Be 
Nice,” and “Be Honest.” The school is distinguished by several extensions to the typical 
school calendar: extended school hours; Saturday school held two weeks out of each 
month; and an extended school year with a four week summer program. 

Memphis Business Academy (MBA). MBA is located in the in the business 
district in the heart of downtown Memphis. Housed in the basement of a church, this 
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charter school’s facility is small and unconventional in comparison to a traditional school 
setting. In its first year of operation, the 2005-2006 school year, the school served 62 
sixth grade students. Almost all (99%) were African American and 72% were eligible for 
free or reduced price lunch. The school employed six full-time teachers, two part-time 
teachers, and one paraprofessional. The student to teacher ratio was 17:1. The school 
incorporates business education, entrepreneurship and financial responsibilities into the 
core curriculum. 

Promise Academy. Promise Academy is located in Memphis and currently 
housed in St. Paul’s Episcopal Church. The 2005-2006 school year was the school’s 
first year in operation, during which only kindergarten classes were conducted. Of the 
53 students served for during the first year, 98% were African American and 91% were 
eligible for free or reduced price lunches. Three full-time teachers, one part-time teacher 
and two paraprofessionals were employed by the school during its first year. The 
student to teacher ratio was 18:1. The school’s mission is “to prepare children to excel 
in the nation's most rigorous high schools. Our central and only work is to teach and 
inspire the mind, body and spirit of our children so that they can succeed in any 
academic or cultural setting.” The educators at Promise Academy want their students to 
learn and excel and want to provide that opportunity in a unique atmosphere that the 
students would not experience in most elementary schools. 

Soulsville Charter School. The Soulsville Charter School is located in a lower 
socio-economic urban area in Memphis, on the same property as the historic Stax 
Museum of American Soul Music, known as Soulsville, USA. In the school’s first year of 
operation, the 2005-2006 school year, 56 sixth grade students were served. All students 
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were African American and 91% qualified for free or reduced price lunch. Four full-time 
teachers were employed during the initial year. The school’s stated mission is “to 
provide music education and exciting performance opportunities designed to enhance 
our students' musical abilities and their academic and leadership skills so they may 
become prosperous citizens and facilitators for community change.” The school 
incorporates a concentration on orchestra and musical skills development into the core 
curriculum developed by Memphis City Schools. The school’s plan includes extended 
school days and half-day Saturday classes. 

Southern Avenue Charter School for Academic Excellence. Southern Avenue 
Charter School for Academic Excellence is located in Memphis and housed at 
Greenwood Christian Methodist Episcopal Church, near the University of Memphis 
campus. During its first year of operation, the 2005-2006 school year, 65 kindergarten 
and first grade students were served. The school plans to expand to include second 
grade for the 2006-2007 year. In addition to the basic curriculum, “kaleidoscope” 
activities such as art, music, ballet and tae kwon do are offered. The school also offers 
Spanish once a week. The school calendar and school days are extended at Southern 
Avenue, with “kaleidoscope” classes beginning at 3:00 in the afternoon. The school 
supplements the students’ education with reading and math tutorials during an after 
school program, which runs from 4:00 pm to 6:00 pm. 

Instrumentation 

Instruments used for data collection were developed and validated by the Center 
for Research in Educational Policy (CREP) as part of the Formative Evaluation for 
School Improvement Process (FEPSI) (see www.memphis.edu/crep ). A description of 
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each instrument is provided below. A summary of the instrumentation employed to 
address each research question is provided in Table 2. The standard data collection 
schedule is shown in Table 3. 

Table 2 

Evaluation Questions by Instrument 





Evaluation Questions 


Instruments 


1 . 


What are the immediate and long-term impacts of the 
charter school implementation on student 
achievement, attendance, and drop-out/attrition rates? 


Analysis of achievement data and archival data 
(attendance, attrition) 


2. 


What is the frequency of usage of various traditional 
and alternative instructional strategies? 

What is the quality of usage of selected instructional 
strategies associated with best practices? 

How do charter school outcomes compare to those 
reflected in national norms? 


SOM 

RSCA 


3. 


What is the school climate at the charter school? 

Does school climate improve over time? 

How do charter school climate outcomes compare to 
those reflected in national norms? 


School Climate Inventory (SCI) 
Teacher Focus Group 
Student Focus Group 
Principal Interview 


4. 


To what degree and levels of quality are the goals and 
strategies of the charter school being implemented? 


Implementation Benchmarks 

Teacher Focus Group 

Principal Interview 

CSTQ 

SOM 

RSCA 


5. 


What are teacher reactions to and experiences in the 
charter school? 

What are the adequacy and quality of professional 
development and resources? 


CSTQ 

Teacher Focus Group 
Principal Interview 


6. 


What are parent (caregiver) reactions to and 
experiences with the charter school? 

To what degree and in what ways are caregivers 
involved with the school? 


Parent Survey 
Principal Interview 
Teacher Focus Group 
Student Focus Group 
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Table 3 

Data Collection Outline 



Instrument 


Timeline 


Description 


Benchmark training, development, 
revision 


Fall 2005 


Train school Principal and Staff; develop draft 
document; refine existing document 


SOM 


Fall 2005; 
Spring 2006 


6 three-hour observations 


Rubric for Student-Centered Activity 
(RSCA) 


Fall 2005; 
Spring 2006 


Same as SOM Data Collection 


Principal Interview 


Early Spring 2006 


1 hour interview 


Charter School Teacher Questionnaire 
(CSTQ) 


Spring 2006 
(Faculty Meeting) 


Teachers complete the questionnaire (part- 
time faculty can also complete) 


School Climate Inventory (SCI) 


Spring 2006 
(Faculty Meeting) 


Professional staff complete inventory (part- 
time faculty/staff can also complete) 


Parent Survey 


Spring 2006 


Parents/caregivers complete the survey 


Teacher Focus Group 


Spring 2006 


1 hour group interview 


Student Focus Group 


Spring 2006 


30-50 minute group interview 


Benchmarking Review 


May 2006 


Principal/Staff review progress 



School Climate Inventory (SCI). The School Climate Inventory (SCI) was 
developed by researchers at the Center for Research in Educational Policy, at The 
University of Memphis in 1989. Since that time, the instrument has been used for 
school-based improvement planning in schools and school districts in several states, 
and has proven to be especially valuable for the monitoring and management of change 
initiatives. 

The SCI consists of seven dimensions logically and empirically linked with factors 
associated with effective school organizational climates. Each scale contains seven 
items, with 49 statements comprising the inventory. Responses are scored through the 
use of Likert-type ratings [strong disagreement (1) to strong agreement (5)]. Each scale 
yields a mean ranging from 1 to 5 with higher scores being more positive. Additional 
items solicit demographic information. 
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Face validity of the school climate items and logical ordering of the items by 
scales were established by the research team during the development of the inventory 
(Butler and Alberg, 1991). Subsequent analysis of responses collected through 
administration of the inventory in a variety of school sites substantiates validity of the 
items. Scale descriptions and current internal reliability coefficients, obtained using 
Cronbach's alpha, appear in Table 4. 



Table 4 

School Climate Inventory Internal Reliability and Scale Descriptions 



Scale 


Internal 

Reliability 


Description 


Order 


a=.8394 


The extent to which the environment is ordered and 
appropriate student behaviors are present 


Leadership 


a=.8345 


The extent to which the administration provides 
instructional leadership 


Environment 


a=.8094 


The extent to which positive learning environments 
exist 


Involvement 


a=.7582 


The extent to which parents and the community are 
involved in the school 


Instruction 


a=.7453 


The extent to which the instructional program is well 
developed and implemented 


Expectations 


a=.7275 


The extent to which students are expected to learn 
and be responsible 


Collaboration 


a=.741 7 


The extent to which the administration, faculty, and 
students cooperate and participate in problem 
solving 



School Observation Measure (SOIVP). The SOM was developed to determine 
the extent to which different common and alternative teaching practices are used 
throughout an entire school (Ross, Smith, & Alberg, 1998). The standard, or whole- 
school, SOM procedure involves observers’ visiting 10-12 randomly selected 
classrooms, for 15 minutes each, during a three-hour visitation period. The observer 
examines classroom events and activities descriptively, not judgmentally. Notes are 
taken relative to the use or nonuse of 24 target strategies. At the conclusion of the 
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three-hour visit, the observer summarizes the frequency with which each of the 
strategies was observed across all classes on a data summary form. The frequency is 
recorded via a 5-point rubric that ranges from (0) Not Observed to (4) Extensively. Two 
global items are used to rate, respectively, the level of academically-focused 
instructional time and the degree of student attention and interest. 

The SOM strategies include traditional practices (e.g., direct instruction and 
independent seatwork) and alternative, predominately student-centered methods 
associated with educational reforms (e.g., cooperative learning, project-based learning, 
inquiry, discussion, technology use as a learning tool). The strategies were identified 
through surveys and discussions involving policy makers, researchers, administrators, 
and teachers, as those most useful in providing indicators of schools’ instructional 
philosophies and implementations of commonly used reform designs (Ross, Smith, 
Alberg, & Lowther, 2004). 

To ensure the reliability of data, observers receive training, a manual providing 
definitions of terms, examples and explanations of the strategies, and a description of 
procedures for completing the instrument. After receiving the manual and instruction in 
a group session, each observer participates in sufficient practice exercises to ensure 
that his/her data are comparable with those of experienced observers. In a reliability 
study (Lewis, Ross, & Alberg, 1999), pairs of trained observers selected the identical 
overall response on the five-category rubric on 67% of the items and were within one 
category on 95% of the items. Further results establishing the reliability and validity of 
SOM are provided in the Lewis et al. (1999) report. In a reliability study using 
Generalizability Theory, Sterbinsky & Ross (2003) found reliability at the .74 level for 5 
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SOMs conducted at a school. Reliability increased to .82 with 8 SOMs and to .85 with 
10 SOMs conducted at a school. 

Rubric for Student-Centered Activities (RSCA). The Rubric for Student-Centered 
Activities was developed by CREP (Lowther, Ross, & Plants, 2000) as an extension to 
SOM. The RSCA is used by observers to more closely evaluate the degree of learner 
engagement in seven selected areas considered fundamental to the goals of increasing 
student-centered learning activities (cooperative learning, project-based learning, 
higher-level questioning, experiential/hands-on learning, student independent 
inquiry/research, student discussion, and students as producers of knowledge using 
technology). These strategies reflect emphasis on higher-order learning and attainment 
of deep understanding of content, and whether or not technology was utilized as a 
component of the strategy. Such learning outcomes seem consistent with those likely 
to be engendered by well-designed, real-world linked exercises, projects, or problems 
utilizing technology as a learning tool. 

Each item on the RSCA includes a two-part rating scale. The first is a four-point 
scale, with 1 indicating a very low level of application, and 5 representing a high level of 
application. The second is a Yes/No option to the question: “Was technology used?” 
with space provided to write a brief description of the technology use. The RSCA was 
completed as part of SOM observation periods. 

To ensure the reliability of data, observers receive training consisting of terms, 
examples, and explanations of the target strategies, and a description of procedures for 
completing the instrument. After receiving the manual and instruction in a group 
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session, each observer participates in sufficient practice exercises to ensure that his/her 
data are comparable with those of experienced observers. 

Charter School Teacher Questionnaire. This questionnaire was designed to 
assess teacher perceptions about the school in the areas of professional development, 
support, pedagogical change, and outcomes. Included on the questionnaire are 20 
closed-ended items using a five-point Likert-type scale (from strongly disagree to 
strongly agree), and the following four open-ended questions: What do you see as 
positive or most successful aspects of your charter school?; What do you consider to be 
negative aspects or areas in need of improvement at your school?; In your opinion, 
what makes a charter school (like this one) different from a regular public school?; and, 
Any other comments you would like to make regarding your experiences as a charter 
school teacher? 

Charter School Parent Questionnaire. This instrument was designed to obtain 
parent perceptions of the school in areas such as instruction, curriculum, 
communication, and opportunities for involvement. Included on the questionnaire are 
17 closed-ended items using a five-point Likert-type scale (from strongly disagree to 
strongly agree), and the following four open-ended questions: What are the strengths of 
this school?; What would you like to see improved at this school?; In your opinion, what 
makes a charter school (like this one) different from a regular public school?; and, Any 
other comments you would like to make about this school or charter schools in general? 

Focus groups and interviews. To supplement the survey data, site researchers 
at each charter school conducted a principal interview, a teacher focus group, and a 
student focus group. The time period for each was approximately 30 - 60 minutes for 
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each interview. Teacher and student participants were randomly selected to participate. 
A semi-structured protocol, involving standard questions with flexibility for follow-up on 
selected responses, was used. In all three protocols the basic questions concerned 
experiences during the year, differences from regular (non-charter) schools, reactions to 
major school components (e.g., teaching methods, curriculum, parent involvement), 
perceived strengths and weaknesses, and recommendations for improvement. 
Procedure 

A “site researcher” from CREP was assigned data collection responsibility for 
each charter school. This individual and support research staff visited the assigned 
school several times during the year to conduct the SOM visits, administer the 
questionnaires, and conduct the interview and focus groups. In addition, the site 
researcher worked with the school leadership team to develop “implementation 
benchmarks” describing beginning, intermediate, and full implementation phases and 
associated evidence indicators for major school components in the areas of curriculum, 
instruction, and organization. At the end of the year, the site researcher met again with 
the leadership team to determine and identify the phase that had been achieved for 
each benchmark. Data from all instruments were then used to prepare individual school 
formative evaluation reports indicating status and progress during the year, as well as 
the present evaluation report. 
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Results 



In this section, the findings from the various data sources are presented. First, 
the findings from the 12 charter schools by cohort are summarized separately for each 
question. The findings are then integrated across schools and cohorts to address each 
of the research questions. 

Question 2. What is the frequency of usage of various traditional and alternative 
(student-centered) instructional strategies in the charter schools and compared to 
national norms? 

Observational data collected via the School Observation Measure (SOM) and the 
Rubric for Student-Centered Activities (RSCA) were used to gauge the frequency with 
which various traditional and alternative instructional strategies were observed in the 
charter schools. More specifically, patterns in results were organized under four 
different categories of instructional orientation and strategies: teacher-centered, 
student-centered, technology usage, and higher-order instruction. Independent 
seatwork was considered a more traditional, teacher directed orientation because 
students are primarily doing work directly prescribed by teachers and often in 
combination with direct instruction. The findings from the observations conducted in the 
charter schools were compared to the national normative data available for the SOM. 
For reference purposes, these norms are provided in Appendix A. In addition to 
comparing the observational data to norms, we also noted trends in the results across 
years for those charter schools in their second and third year of operation. 
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Cohort 1 



Circles of Success Learning Academy. COSLA continued to rely on direct 
instruction as the primary means of instructional delivery. Across classroom 
observations, direct instruction was recorded as frequently or extensively used 100% of 
the time. The percentage of time devoted to direct instruction increased from the 
previous year and is higher than norms for elementary schools nationally. Compared to 
the previous year, current findings indicate more frequent use of independent seatwork. 
This strategy was used occasionally or frequently about 83% of the time, but this 
percentage is comparable to the national norms. Student-centered activities were rarely 
observed. There was a decline in the overall use of cooperative or collaborative 
learning. However, a small percentage of time, somewhat strong applications of 
cooperative learning, experiential hands-on learning, and student discussion were seen. 
Technology use was not observed. Higher order questions were occasionally used 
50% of the time, but the frequency of use was lower than in previous years. There was 
a somewhat strong application of higher-level questioning observed in some cases. 
Higher-level instructional feedback was occasionally observed about one-third of the 
time, but the prevalence of this strategy was lower than in previous years or when 
compared to national norms. 

Memphis Academy of Health Sciences. The rates of traditional, teacher- 
centered approaches to instruction have been consistently high across years. 
Increases in direct instruction and independent seatwork were observed in the most 
current year, with these strategies observed frequently or extensively 100% of the time. 
These rates exceeded levels observed in secondary schools nationally. In contrast, 
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declines in most student-centered and higher-order strategies were noted. For 
example, the incidence of cooperative learning decreased, as did the use of higher-level 
instructional feedback. It appears that the gains noted in more alternative or innovative 
strategies in the second year were reversed in the third year of operation. The 
incidence of student-centered and higher-order strategies tended to be lower than those 
reported for secondary school nationally. RSCA data confirmed this trend and indicated 
low intensity or weak application of these strategies. There was also a decline in the 
use of computers for instructional delivery, but the frequency of technology used as a 
learning tool was identical across years. Considering both categories of computer use, 
the rates observed at MAHS did not compare favorably to the national sample. 
Technology was not used to support the types of student-centered or higher-order 
instruction measured on the RSCA. 

Memphis Academy of Science and Engineering. As with the two preceding 
Cohort 1 schools, the instructional strategies observed at MASE remained highly 
traditional and teacher-centered across the three years of operation. The percentage of 
time devoted to direct instruction remained high in the third year and was somewhat 
higher than the normative data for secondary schools. Direct instruction was observed 
frequently or occasionally more than 80% of the time. Student engagement in 
independent seatwork was used extensively (50%) or occasionally (50%), but the 
prevalence of this strategy is also high in secondary schools nationally. One exception 
to this trend was the marked increase in teacher acting as a coach or facilitator across 
years, which suggests more student-centered activities. There was also an increase in 
the use of student discussion compared to previous years, but the move was from rare 
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or no use to occasional use. The prevalence of student discussion was higher than 
national norms, but the relatively low frequency with which other student-centered 
strategies were observed was similar to those found in the normative sample. 
Furthermore, the application of student-centered strategies remained limited in intensity 
or quality, as evidenced by RSCA data. Technology use was occasionally observed in 
about one-third of the classroom observations, and in a few cases the application of 
students as producers of knowledge using technology was strong. MAHS has 
somewhat higher prevalence of technology use than secondary schools nationally even 
though usage is not frequent in an absolute sense. The frequency of higher-order 
feedback and questioning also surpassed national norms, yet the application of these 
strategies was mostly limited. Across years, the prevalence of higher-order instructional 
strategies was consistent. 

Smithson-Craighead Academy. An increase in direct instruction at SCA was 
observed. Direct instruction was observed frequently or extensively 100% of the time 
and these rates are higher than national norms. However, use of independent seatwork 
was also frequently observed but less extensively than in previous years and at 
somewhat lower rates than national norms. A declining trend was also noted for the 
use of higher-order strategies, but the rates were similar to those observed in the 
normative sample. For instance, higher-level instructional feedback was used at least 
occasionally during all of the visits in the first year, but this percentage dropped to 33% 
of the time for the third year. There was a similar drop in the use of higher-level 
questioning, both relative to prior years and to national norms. There were gains in the 
use of some student-centered strategies, but such occurred in the rare or occasional 
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categories, not the frequent or extensive categories. Although the observed 
frequencies of some student-centered strategies showed increases, RSCA data reflect 
a decline in both frequency and quality of application. The prevalence of technology, 
although low, compares favorably to normative data and remained consistent across 
years. 

Cohort 2 

City University School of Liberal Arts. There was a notable drop in the 
prevalence of direct instruction, the most common type of teacher-directed orientation. 
It was employed either frequently (50%) or occasionally (50%), but not extensively in 
any of the observed class sessions. These figures are comparable to national norms 
for secondary schools. One might expect that a reduction in teacher-centered 
orientation might signal a gain in student-centered orientations, but this was not the 
case. Student-centered strategies were typically rarely or not observed, as was also the 
case last year. The most frequent instructional strategy was independent seatwork, and 
its usage rates were comparable to national norms. Both the SOM and RSCA indicated 
little change in the prevalence or quality of student-centered or higher-order instructional 
strategies. In contrast to last year when no technology use was observed, rare, lower 
than normative use of technology was recorded this year. 

Star Academy. Although there was an increase in the percentage of time in 
direct instruction compared to last year (from 50% to 83% in the extensively observed 
category), the prevalence of many student-centered strategies also increased. 
Specifically, there were notable increases in use of cooperative or collaborative learning 
as well as teachers acting as coach or facilitator and work centers. The percentage of 
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time spent in direct instruction and in many types of student-centered instruction was 
higher than the national norms obtained for elementary schools. There was a slight 
decrease in the use of independent seatwork, but the prevalence was still higher than 
national norms. Higher-order strategies were more frequent when compared to the 
previous year and surpassed those reported in the normative sample. Furthermore, 
when these strategies were employed, RSCA findings indicate somewhat strong or 
strong application. However, the use of technology remained elusive during the second 
year of operation. 

Yo! Academy. Observational findings were characterized by a decrease in 
teacher-centered instruction that corresponded with an increase in higher-order 
instructional strategies. Although direct instruction and independent seatwork were still 
frequently observed, the rate declined when compared to last year and was more 
comparable to secondary level norms. The rate of higher order instruction, most 
notably the use of higher level questioning strategies and feedback, increased to levels 
that surpassed those reported for the national sample. RSCA evidence reflected 
somewhat strong or strong application of higher-level questioning about 30% of the 
time. The findings for student-centered instruction were somewhat mixed. Compared to 
last year, there were increases in experiential learning, sustained writing, project-based 
learning, and cooperative learning, but the changes were largely represented among 
the not observed, rarely or occasionally observed categories. The incidence of 
technology use was nearly identical to the previous year’s findings and was lower than 
those reported nationally. 
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Cohort 3 



KIPP Academy Nashville. In their first year of operation, instruction at KIPP 
academy was primarily teacher-centered in the form of direct instruction. This 
orientation was observed frequently or extensively in over 80% of the classroom visits, 
and this percentage is higher than the national elementary school norms. However, 
independent seatwork, another traditional instructional strategy, was less frequent 
compared to the normative sample. Instruction reflected occasional to frequent use of 
higher-order strategies that exceeded national norms. Notably, higher-level questioning 
and feedback were frequently or occasionally observed the majority of time and the 
level of application was judged to be somewhat strong or strong in over 40% of the 
cases. Student-centered strategies were less frequently or occasionally observed, but 
the frequencies tended to be higher than levels reported nationally. For example, 
sustained writing and student discussion were occasionally observed during half of the 
schools visits and project-based learning was occasionally observed one third of the 
time. The implementation of student-centered strategies was considered strong or 
somewhat strong in several cases. There was no technology use observed. 

Memphis Business Academy. The frequencies across categories tended to be 
lower than national norms in most categories. Direct instruction, the most commonly 
observed orientation at MBA, was frequently observed about half of the time. 
Independent seatwork occurred frequently or extensively in about one third of the 
observed sessions. Student-centered strategies and technology use were either not 
observed or rarely observed. The only exception was the teacher acting as a coach or 
facilitator, which occurred occasionally or frequently one third of the time. Higher level 
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questioning and feedback occurred occasionally or rarely with rates comparable to the 
national norms. The RSCA findings suggested limited to strong application of these 
higher-level strategies in some cases. 

Promise Academy. Traditional orientations that included direct instruction and 
independent seatwork were predominantly observed at Promise Academy. Direct 
instruction was frequently or extensively observed 100% of the time, and independent 
seatwork was frequently observed over 80% of the time. Both levels were higher 
national elementary school norms. Teacher acting as coach or facilitator was the most 
commonly seen student-centered strategy, with a frequency exceeding national norms. 
However, cooperative and experiential learning were occasionally observed at 
normative levels. Higher-order strategies were occasionally observed over half the time, 
which is also similar to the norms. In about 10% of the cases, there was at least 
somewhat strong application of higher-level questioning and experiential learning. 
Technology use was observed only rarely or occasionally at levels lower than observed 
in the national sample. 

Southern Avenue Charter School of Academic Excellence. A mix of both teacher 
and student-centered orientations was observed at this charter school. Although direct 
instruction was most frequently observed more than 80% of the time, experiential 
learning, work centers, and teacher acting as coach/facilitator were frequently to 
extensively observed at least half the time. The percentages reported in these 
categories were higher than the national norms. In terms of quality or strength of 
application, strong or somewhat strong application was noted for cooperative and 
experiential learning in at least 10% of the cases. It is notable that the incidence of 
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independent seatwork was lower than observed nationally. There was some evidence 
of technology use in the classrooms. Technology used as a learning tool or resource 
was occasionally to frequently observed 50% of the time, a level that surpassed national 
norms. There was less indication of the use of higher order strategies when considering 
absolute frequencies or in reference to the normative data. 

Soulsville Charter School. When compared to national elementary school norms, 
Soulsville Charter School had somewhat higher levels of direct instruction and 
independent seatwork. Observers rated the usage of direct instruction as frequent or 
extensive in nearly 85% of the cases. Independent seatwork was rated as occasionally 
present 100% of the time. In contrast, student-centered strategies were observed less 
frequently. Only teacher acting as coach or facilitator was frequently observed. Other 
student-centered strategies occurred at lower levels in both an absolute sense and 
compared to national norms. Encouragingly, frequent use of higher-order questioning 
and feedback was more apparent and at levels that surpassed the normative sample, 
particularly for feedback. RSCA findings indicated somewhat strong application of 
higher-level questioning strategies. Technology use was more common, with 
technology utilized as a learning tool occasionally observed 50% of the time, a rate 
more frequent than national norms. However, computers used for instructional delivery 
was less common. 

Summary findings across schools 

Table 5 presents the summary findings by school in four major categories 
associated with instructional orientation and strategies. Across schools and cohorts, 
teacher-centered instruction was the dominant orientation. It should also be noted that 



139 




the percentage of time spent in direct instruction is high in both the elementary and 
secondary school classrooms nationally. There were two schools, Cityll and MBA, 
rated as moderate because the findings suggested less extensive or frequent use of 
both direct instruction and independent seatwork at levels lower than those reported 
nationally. It was somewhat surprising that somewhat less traditional instruction was 
observed for MBA in it first year of implementation. It was also interesting that less 
teacher-centered orientations did not correspond to more student-centered instruction in 
these two schools. 

Although a wide array of student-centered strategies was observed across 
charter schools, most schools were not considered to be strong in this category. The 
two exceptions were Star and Southern, both of which are elementary schools. At Star, 
cooperative learning was combined with teacher acting as coach and facilitator, 
whereas at Southern, experiential learning and work centers were coupled with the 
teacher acting as coach or facilitator. In both cases, the teacher role was apparently 
less directive in the midst of more student-centered activities. One might expect that 
more student-centered activities would be prevalent among first cohort charter schools 
that were in their third year of implementation, but this was not the case. The 
anticipated trend that charter schools might move from more traditional to more student- 
centered strategies was not completely realized because many schools were rated as 
moderate or weak in this category. However, the charter schools, much like traditional 
schools, are under heavy pressure to perform well on the state achievement 
assessments. The schools, therefore, may have good intentions of using student- 
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centered approaches but rely increasingly on teacher-directed approaches to ensure 
coverage of the broad curriculum tested. 

The pattern of findings was similar for the category of higher-order instruction. 
That is, most schools were rated as either weak or moderate in this category and more 
extensive implementation of these strategies did not occur in early cohort schools. As 
is the case with other orientations or instructional strategies, national norms do not 
reflect extensive levels of higher-order instruction. Only two schools rated as strong in 
this category (Yo! and KIPP), which were second and third cohort schools respectively. 
In addition, findings at these schools reflect stronger or higher quality of application in 
some instances. 

Technology usage was the least prevalent category of instruction across charter 
schools. In some schools no technology use was observed, whereas in others 
technology was rarely used to support learning. No charter schools were considered to 
be strong in this area, and only four were rated as moderate (MAHS, MASE, Southern, 
and Soulsville). Again, there was no apparent trend by year of implementation. In 
many cases the levels of technology use did not compare favorably with the normative 
data. 
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Table 5 

Summary Ratings for Schools on Teaching Orientations 



School 



Teacher- Student- Higher-order Technology 

centered centered strategies usage 




Beginning/Low Usage 



Moderate 



Strong 
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Question 3. What is the school climate at the charter schools and how does the climate 



compare to national norms ? 

To address this research question, we primarily relied on the rating scale data 
obtained from the School Climate Inventory (SCI). Because we compared the SCI 
results for the charter schools with national norms, the national averages for elementary 
and secondary schools (middle and high schools) are presented in Table 6. The 
interpretation of SCI data was augmented by the qualitative responses obtained from 
focus group interviews, principal interviews, and open-ended items on the SCI. Any 
themes related to the dimensions of school climate were identified and used to 
illuminate the quantitative findings. 

Table 6 

School Climate Inventory (SCI-R) Dimension Averages and Standard Deviations for Elementary 
and Secondary Schools 



Dimension 


COSLA 


SCA 


Star 


KIPP 


Promise 


Southern 


Elementary 

Norm 


Collaboration 


4.50 

(.71) 


4.54 

(.63) 


4.49 

(.85) 


4.60 

(.55) 


4.29 

(.85) 


4.29 

(.60) 


3.88 (.61) 


Environment 


4.31 

(.93) 


4.42 

(.77) 


4.54 

(.69) 


4.51 

(.78) 


4.68 

(.55) 


4.43 

(.63) 


3.96 (.67) 


Expectations 


4.66 

(.54) 


4.76 

(.43) 


4.82 

(.39) 


4.89 

(.32) 


4.82 

(.39) 


4.50 

(.51) 


4.04 (.60) 


Instruction 


4.51 

(.78) 


4.61 

(.59) 


4.71 

(.58) 


4.54 

(.61) 


4.71 

(.71) 


4.54 

(.58) 


4.17 (.50) 


Involvement 


4.60 

(.65) 


4.36 

(.85) 


4.51 

(.82) 


3.79 

(.74) 


4.39 

(.92) 


4.54 

(.58) 


3.91 (.58) 


Leadership 


4.23 

(.88) 


4.82 

(.45) 


4.86 

(.35) 


4.40 

(.74) 


4.43 

(.84) 


4.50 

(.51) 


4.08 (.70) 


Order 


4.23 

(.94) 


3.70 

(1.31) 


4.48 

(.74) 


4.43 

(.61) 


4.04 

(1.20) 


3.96 

(.64) 


3.50 (.80) 


OVERALL 


4.43 

(.79) 


4.46 

(.83) 


4.63 

(.67) 


4.45 

(.69) 


4.48 

(.84) 


4.39 

(.60) 


3.93 (.55) 
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Table 6 (continued) 



Dimension 


CityU 


MAHS 


MASE 


Yo! 


MBA 


Soulsville 


Secondary 

Norm 


Collaboration 


4.45 

(.74) 


3.89 

(.78) 


4.25 

(.84) 


4.46 

(.66) 


4.18 

(.67) 


3.43 

(1.26) 


3.71 (.66) 


Environment 


4.43 

(.70) 


3.69 

(.78) 


4.41 

(.65) 


4.44 

(.65) 


3.86 

(.57) 


3.43 

(1.20) 


3.73 (.71) 


Expectations 


4.57 

(.70) 


4.11 

(.92) 


4.65 

(.59) 


4.49 

(.78) 


3.93 

(.90) 


4.43 

(.51) 


3.82 (.64) 


Instruction 


4.57 

(.74) 


4.25 

(.72) 


4.40 

(.73) 


4.32 

(.68) 


3.95 

(.86) 


4.00 

(.77) 


4.06 (.51) 


Involvement 


4.34 

(.76) 


3.90 

(.88) 


4.32 

(.74) 


4.27 

(.78) 


4.21 

(.83) 


3.39 

(1.13) 


3.63 (.61) 


Leadership 


4.86 

(.36) 


4.15 

(.87) 


4.41 

(.76) 


4.44 

(.67) 


3.75 

(.75) 


3.86 

(.65) 


3.94 (.72) 


Order 


4.21 

(.74) 


3.50 

(1.06) 


4.29 

(.89) 


4.18 

(.77) 


2.46 

(.79) 


4.00 

(.95) 


3.26 (.87) 


OVERALL 


4.49 

(.70) 


3.93 

(.90) 


4.39 

(.76) 


4.37 

(.72) 


3.76 

(.96) 


3.79 

(1.04) 


3.73 (.59) 



Cohort 1 

Circles of Success Learning Academy. School climate remained a definite 
strength at COSLA. The mean ratings ranged from 4.23 to 4.66 and well exceeded the 
elementary school norms on each dimension. The means obtained this year were 
slightly lower than last year’s means but may be attributable to the difference in a 
reduction in the sample size (from 10 to 6). The gain in mean score on the order 
dimension was maintained when compared to year one (3.86 to 4.23). 

Qualitative interview data supports the quantitative findings indicative of a 
positive school climate. Comments from the teachers, students, and principal were 
overwhelmingly favorable with respect to climate. For example, teachers characterized 
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the climate as being “warm, safe, nurturing, close-knit, and homey.” With respect to 
leadership, there was mention of the difficulty of adjusting to a new principal, yet 
teachers described her as supportive and open-minded. Another concern related to 
leadership was the lack of mentoring for teachers and not having a curriculum 
coordinator. Other somewhat negative comments were related to school environment 
due to space limitations and concerns. However, there are plans to move to a new 
building next year, which promises to alleviate these concerns. 

Memphis Academy of Health Sciences. School climate scores continued to 
decline across years on the first four dimensions (Collaboration, Environment, 
Expectations, and Instruction). The means on these dimensions ranged from 3.69 to 
4.25 and were similar to or higher than the norms obtained for secondary schools. In 
contrast, there were slight gains in mean scores on the remaining 3 dimensions 
(Involvement, Leadership, and Order) when compared to last year, yet these scores 
were still lower than those obtained in the initial year of operation. As was true in 
previous years, the highest mean rating was obtained for the Instruction dimension 
(4.25), and the lowest was for the Order dimension (3.50). An examination of 
responses to individual items comprising the Order dimension showed that the lowest 
levels of agreement were obtained for items related to student misbehavior and 
tardiness. 

The qualitative data mostly suggest a positive school climate. Students 
expressed appreciation for the small school environment that affords more individual 
attention from teachers. Although students said the long hours were hard, they 
recognized that it provided them more time to learn. The principal described students 
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as “excited to be here and enjoy school more even though we press them on their 
work.” Teachers commented that the climate was very supportive and that they were 
“invested in each others’ success.” Their portrayal of the principal was also very 
positive. “The principal is the most supportive and knowledgeable we have worked 
with.” The decline in mean ratings on some dimensions across years may be due to the 
high demands placed on teachers and the fact that new teachers were still adjusting the 
program and school. 

Memphis Academy of Science and Engineering. Quantitative data indicate an 
increasingly positive school climate at MASE. The mean ratings across SCI dimensions 
continued to be high in the third year of implementation and show some improvement 
when compared to the second year. The mean values ranged from a low of 4.25 for 
Collaboration to a high of 4.65 for Expectations. The largest gains were seen on the 
dimensions of Environment (from 4.06 to 4.41), Involvement (from 4.06 to 4.32), and 
Order (from 3.68 to 4.29) dimensions. Overall, the mean ratings obtained for MASE 
were uniformly higher than those reported in the national sample of secondary schools. 

The qualitative data help explain the increasingly positive school climate at 
MASE. The principal commented that the second year could be characterized as 
“disconnected,” with better discipline but less teacher support. He maintained it has 
been the “best year for how the staff and students relate to one another.” When 
students were asked what they liked best about their school, they specifically mentioned 
the teachers and instruction as well as the safe yet friendly environment. However, 
students also made a few negative comments related to student misbehavior that 
adversely affected all students. In contrast, teachers offered positive evaluation of 
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MASE with respect to school climate. They appreciated the safe learning environment 
and the support from their principal. “It is well structured, organized, and has high 
standards for students.” 

Smithson-Craighead Academy. Third-year findings showed a marked 
improvement in the school climate at SCA. The overall mean rating increased from a 
3.92 reported in the second year to 4.46 in the most recent year. Mean increases from 
the second to third year were observed on each of the seven dimensions. The highest 
scores were obtained for Leadership (4.82) and the lowest scores continued to be 
observed for Order (3.70). With the exception of the mean rating obtained for order, the 
averages were also somewhat higher than those reported in the initial year of 
implementation. These dimension means well exceeded those reported in the national 
sample of elementary schools. 

Although many interview responses were indicative a positive school climate, 
some comments suggested that student discipline and behavior problems continued to 
be a problem at SCA. Teachers estimated that about half of their students have special 
needs, have Individual Education Plans (lEPs) and take medication. Because students 
arrive at SCA without basic academic skills and have some behavior or social problems, 
instruction and discipline are more challenging. Students also mention behavior 
problems when asked about what they don’t like about their school. On a more positive 
note, the principal asserted that their greatest accomplishment this year is the children’s 
enthusiasm toward learning. Teachers feel a sense of accomplishment by impacting 
the lives of their students. Likewise, the students appear to like and appreciate their 
teachers. 
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Cohort 2 



City University School of Liberal Arts. School climate improved at City U during 
its second year of operation. Although the SCI ratings were high in the first year, mean 
score gains were observed across all seven dimensions. The overall mean was a 4.49 
compared to the 4.10 mean rating obtained last year. The highest mean rating was 
observed for Leadership, while the lowest mean was obtained for Order (4.21). The 
ratings well surpassed those reported for the national sample of secondary schools, but 
the sample size at City U was very small (n= 6). 

Although the quantitative data clearly indicate a very positive school climate, the 
qualitative findings are mixed. The teacher and principal interview responses were 
generally favorable in tone. The principal described the teachers as collaborative and 
supportive of the school as well as one another. “The teachers have bonded this year 
and have a shared sense of mission.” He noted that teachers still complain about the 
long hours but “no longer feel burnt out form the discipline problems and overcrowded 
classrooms.” Teachers made similar comments, remarking that the climate was good. 
They appreciated the rapport with administration as well as the collaboration among 
teachers. Although the teachers and principals made favorable comments about the 
impact of the educational program on students, the students themselves were less 
positive. In their focus groups, responses were mainly related to the physical building 
and lack of resources. For example, students complained about the crowding, the 
absence of extracurricular activities, inefficient air conditioning, the cafeteria food, the 
dress code, and not a having a gym, library, or computer lab. However, they also had 
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some positive things to say about City U as a school, most notably the safe environment 
and caring, helpful teachers and administrators. 

Star Academy. School climate continued to be a strength at Star. The mean SCI 
ratings were nearly identical to those obtained in the previous year and clearly 
exceeded national norms. The overall rating this year was a 4.63 compared to a 4.69 
recorded in the previous year. The Leadership dimension had the highest mean rating 
(4.86) and Order had the lowest mean (4.48). These high scores reflect nearly 
unanimous levels of agreement across all items and scales. 

The interview responses clearly support the rating scale data. Principal, 
teacher, and students were very laudatory and supportive of their academy. A recurring 
theme in student comments was that they felt challenged and there were high 
expectations for their success. They appreciated the instructional activities, their 
teachers, the resources available, and the school’s physical environment. A theme that 
emerged in teacher responses was the support they received from the principal, the 
commitment and collegiality among teachers, and the sense of pride and value of 
education instilled among students. Teachers talked about “structure” in a “caring 
atmosphere of learning.” The principal asserted that “the school has been a life 
changing experience for the students”, who seem excited about school most of the time. 
She views the continued focus on mission and goals and the positive relationship 
between teachers and parents as their greatest accomplishments this school year. 

Yo! Academy. Quantitative ratings suggest a notable improvement in school 
climate at Yo!. There was a marked increase in mean ratings from the prior year on 
each of the seven dimensions. The largest gains in mean scores were observed for the 
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Environment (from 3.80 to 4.44), Involvement (from 3.60 to 4.27), and Order (from 3.89 
to 4.37) dimensions. Overall the mean score increased from a 3.89 to a 4.37. The 
mean values were all well above secondary school norms. 

Interview responses illuminate the positive perceptions of school climate 
measured by the SCI. Results across respondent groups point to a caring, supportive, 
and safe environment that maintains high expectations of students. Students 
complimented the small school environment that fostered a “family-like” yet orderly 
atmosphere in which teachers can afford to provide individual attention. The principal 
described Yo! as a “safe haven” and a “safety net” that protects students from some of 
the risks and problems inherent in their larger urban community. He credits teachers’ 
demeanor and attitudes as another contributor to the positive school climate. “The 
school gives teachers a sense of hope, of believing again that students want to be 
taught. Teachers believe they can make a difference.” Teachers concurred and noted 
that “all teachers are committed to the vision of the school.” They complimented the 
support provided from their administrators. They appreciated being treated like 
professionals with integrity who have autonomy in their classrooms. Another recurring 
thread among teacher comments was the high expectations and even admiration they 
held for their students. Last year the teachers were somewhat critical of the 
administration, but this dissatisfaction was not apparent in the current year. One 
possible explanation for the more positive school climate is the turnover in faculty and 
staff. Most of the faculty are new this year with only two teachers returning. The 
assistant principal, counselor, and cafeteria manager were also new to the school this 
year. 



150 




Cohort 3 



KIPP Academy Nashville. The data available suggest a favorable school climate 
for KIPP during its initial year of operation. Although the sample size was small (n=5), 
the ratings point to positive perceptions of climate in this academy. The overall mean 
rating was 4.45, and the mean ratings for the other dimensions ranged from low of 3.79 
for Involvement to a high of 4.89 for Expectations. With the exception of the mean 
rating obtained for Involvement, dimension means were higher than those reported in 
elementary school national norms. An examination of the percentages of agreement to 
individual items that comprise the Involvement dimension reveal that parent 
participation is perceived to be low at KIPP. Only 40% agreed that parents actively 
support school activities, are invited to serve on school advisory committees, and are 
used as volunteers. 

Qualitative data shed some light on the pattern of quantitative findings. 
Regarding Involvement, the principal admitted that parent outreach was their “weakest 
point this year.” Other comments mirrored the quantitative ratings suggesting a 
favorable climate across dimensions. The principal described a school environment 
“where kids feel safe, where they dream big, and where they work hard to make those 
dreams a reality.” Several student comments focused on school characteristics related 
to order and safety. They talked about having strict rules governing student behavior, 
which were viewed as both positive and negative aspects of their school. They also 
noted that there were clear expectations for academic success supported by caring 
teachers. Teachers described their school climate as safe with a strong culture for 
learning. Another teacher comment highlighted the collegiality and support evident at 
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KIPP: “Everyone at this school works very closely together with strong support from the 
administrator and from the teachers for the school mission.” 

Memphis Business Academy. The available quantitative findings reveal large 
variance in ratings across the seven dimensions, possibly influenced by the small 
sample size (n= 4). The overall mean rating of 3.76 was essentially identical to the 
overall national mean for secondary schools. Positive school climate was more evident 
for Involvement (4.21) and Collaboration (4.18). The means obtained on these 
dimensions were notably higher than those reported nationally. The lowest mean 
ratings were obtained for the Order (2.46) and Leadership (3.75) dimensions. Although 
both mean ratings were lower than the norms, the difference was particularly large for 
the Order dimension. An examination of results by individual items suggests that 
student behavior, discipline and absence or tardiness were particularly problematic at 
MBA. 

During the interview, the principal was very candid and thoroughly described both 
the strengths and limitations evident in their initial year of operation. One strength, also 
reflected in the SCI data, centered on a small school atmosphere that promotes a 
culture of learning and affords the students more individual attention. Another strength 
identified was teacher support and commitment. These strengths were also cited during 
the teacher and student focus groups. The adjustment to long school hours within a 
structured environment was difficult for the students and teachers, and some discipline 
problems were apparent. Some students were not prepared for the level of coursework 
or the disciplinary policies at MBA. The principal commented that their “biggest 
challenge has been accepting the fact that many of our initial students were “typical 
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MCS students” in that they arrived academically under-prepared, were “disrespectful”, 
and lacked the social and familial support critical for learning. Order at MBA was a 
challenge during its initial year. 

Promise Academy. The school climate at Promise appeared to be favorable. 
Although the response rate was very low (n= 4), the ratings were consistently high and 
above those reported in normative sample of elementary schools. The overall mean 
was 4.48 and ranged from high of 4.82 for Expectations to a low of 4.04 for Order. The 
ratings indicate very high rates of agreement across items and dimensions that assess 
school climate. 

The largely positive interview responses support the rating scale results. 
According to the teachers, students “really enjoy being at school and love coming each 
day.” The principal concurred, stating that students “feel safe at school and receive lots 
of love and attention.” The principal and teachers agree that the climate is also positive 
for teachers. They described it as a supportive, nurturing environment in which 
teachers have bonded and work as a team. 

Southern Avenue Charter School of Academic Success. The quantitative data 
suggest a positive school climate for this new charter school. It should be noted, 
however, that only 4 individuals completed the survey. The mean SCI ratings ranged 
from a low of 3.96 for the Order dimension to a high of 4.54 observed for both the 
Instruction and Involvement dimensions. The overall mean rating was 4.39. The results 
compare favorably to normative data, with mean scores that exceeded the national 
overall and dimension means. 
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The interview results help confirm the SCI ratings. Two themes apparent in the 
principal and teacher interviews were parent involvement and support for instruction. 
The principal stated, “We have included the parents and we have wonderful parent 
support.” With regard to teacher support, the principal commented that, “we try to get 
the materials and the things they need to be successful and to empower them to teach.” 
According to the teachers, the small school context facilitates cohesion among teachers 
and students. As a first year school, there have been challenges in developing 
programs and policies, but teachers support the school’s mission and educational 
program. Students seem to appreciate the extracurricular activities, such as art, music, 
and karate. 

Soulsville Charter School. Although the sample size was small (n= 4), the SCI 
ratings suggest that school climate at Soulsville during the initial year was moderate. 
The overall mean of 3.79 was virtually identical to the national secondary school norm 
(3.73). Whereas the mean ratings obtained on four of the seven dimensions 
(Collaboration, Environment, Involvement, and Leadership) were somewhat lower than 
national norms, those on the Expectations and Order dimensions were markedly higher 
than the norms. The lowest mean rating was obtained for the Involvement dimension 
(3.39) and the highest for Expectations (4.43). 

Perceptions of school climate differed among response groups. The principal’s 
description of the climate was more positive than those of the teachers and students. 
He stated that students have grown academically and socially, and that the school has 
high levels of community and parental support. The changes in staff were viewed as 
the biggest challenge. Although teachers stated that the students seemed “to be 
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happy” and more motivated to achieve academically, the strict disciplinary procedures 
and lack of physical activity were limitations. According to the teachers, administration 
did not provide enough support for teachers or encourage parent participation. They 
describe their own morale as low. “The biggest challenge for the school during the first 
year of operation has been maintaining staff morale and the long hours with no 
substitutes or staff support available.” The students concurred that the long school 
hours were a difficult adjustment. They also had mainly negative reactions to the strict 
discipline policy that employed fines for misbehavior. As one student put it, “The 
hardest things to get used to were the hours, discipline, fines, getting home late.” The 
low SCI ratings on the Order dimension may reflect negative reactions to the strict 
disciplinary rules and the qualitative data may reflect some difficulty in adjustment 
among students. 

Summary findings across schools. Table 7 depicts the summary results based 
on SCI ratings and qualitative data collected for each of the 12 charter schools 
organized by cohort. Overall, school climate remains a definite strength of these charter 
schools. The quantitative SCI data indicated that all but two schools have a strong 
school climate. The exceptions were MBA and Soulsville, both third cohort schools in 
their initial year of implementation. These schools had somewhat lower mean ratings 
when compared to the normative samples. It should be noted, however, that the 
sample size was very small, especially among the 3 rd cohort schools. The qualitative 
data also suggested a favorable school climate for 9 of the 12 schools that also had 
high mean ratings. The only exception was SCA, which was considered strong in 
climate based on the quantitative ratings but only moderate based on the interview 
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findings. As reported last year, student behavior and order continue to be a challenge 
at SCA due to the large population of special needs students at this charter school. 

With respect to other trends across time, there was an improvement in school 
climate when compared to the previous year’s findings. Among first cohort schools, a 
drop in SCI ratings were observed from the first to the second year, but in the third year 
the scores rebounded to levels equal to or above those reported in the initial year. 
Interview responses were also more positive. Improvements in school climate indices 
were also noted among second cohort schools. The fact that school climate was 
already high compared to the norms and continue to improve bodes well for the success 
of charter schools, particularly if improvement in climate is ultimately linked to 
improvement in student success. 
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Table 7 

Summary Results for School Climate 



School 



Low 



School Climate Qualitative 
Ratings Responses 




Moderate/Average 



Strong 
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Question 4. To what degree and levels of quality are the goals and strategies of the 
charter school being implemented? 

Data obtained from implementation benchmarks, the teacher questionnaire, 
teacher focus groups, principal interview, and classroom observations (SOM and 
RSCA) were used to address this evaluation question. To facilitate summarization of 
findings, the goals and strategies were divided into five major categories: curriculum, 
instruction, organization (leadership, governance), support (resources and professional 
development), and evaluation (accountability, assessment). See Table 8 for summary 
findings. 

Cohort 1 

Circles of Success Learning Academy. Given that COSLA is in its third year of 
implementation, the designation of most benchmarks in intermediate and final phases of 
development were not surprising. In benchmarks related to both curriculum and 
instruction, the school would seem to be Phase II. According to the principal, they are 
still in the process of selecting and fully implementing programs in their various subject 
areas. Though they evaluated their literacy program to be strong, they have made 
changes in their curriculum and are considering other programs. SOM and particularly 
RSCA data indicate that instructional benchmarks may not be fully implemented. More 
student-centered instructional strategies and the integration of technology would 
advance efforts in this category. Some of the instructional strategies may change 
depending on the particular curricular programs adopted. In the remaining three 
categories of organization, support, and evaluation, COSLA was at more advanced 
stages of goal implementation. Based on teacher perceptions, one exception to this 
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trend may be the lack of support from state and local educational agencies and the 
need for more resources to support instruction. 

Memphis Academy of Health Sciences. The evidence from various sources 
points to intermediate stages of development for most benchmarks. The leadership 
team also placed most benchmarks at similar levels of implementation. Notable 
progress has been made in the area of support, but some improvement in the areas of 
support and resources are needed. Parent involvement was judged to be moderate and 
teachers identified the need for more professional development, sufficient planning time, 
technological resources, or other educational materials. Support and organizational 
structure was also affected by the lack of effective assistance from state and local 
educational agencies. Although the leadership team placed their school at the Phase III 
on benchmarks related to curriculum and instruction, the current findings suggest more 
progress toward implementation is still warranted. The SOM and RSCA data indicate 
some decline in the use of student-centered, higher order strategies when compared to 
the previous year’s data. It appears that closer alignment between the curriculum and 
instruction is needed. Beyond having a challenging curriculum, it is not entirely clear 
from the benchmark documents or evaluation reports what specific components or 
materials contribute to the educational program at MAHS. Surprisingly, there is little 
detail or explanation about the integration or focus on health sciences. Evaluation and 
assessment remain strong components of the educational program at MAHS. 

Memphis Academy of Science and Engineering. The leadership team judged 
most benchmarks to be in Phase II. The present findings also suggest that 
implementation of most benchmarks are in the intermediate phases of development. 
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However, the leadership team may have been too optimistic in their evaluation of 
progress made in the category of instruction. The Phase III designation on benchmarks 
related to the application of higher-order thinking skills, subject integration, and the use 
of technology was not supported by SOM and RSCA data. However, in interviews, both 
the principal and teachers described more technology integration exemplified by the 
provision of laptop computers for students. As was true in previous years, it was difficult 
to track progress in assessment and evaluation due to the absence of benchmarks in 
these areas. However, nearly all teachers agreed that goals were reviewed regularly to 
evaluate progress. The leadership team might consider adding benchmarks related to 
evaluation, assessment, and accountability in the next version of the document. 

Smithson-Craighead Academy. Last year, the majority of benchmarks were 
evaluated to be in the initial stages of implementation. This year, the leadership team 
has designated all benchmarks to be the in Phase III, and substantial progress appears 
to have been achieved in the areas or curriculum, organization, and evaluation. 
However, the current findings do not support all designations as in the final stage of 
development. The evidence suggests more moderate ratings in the areas of support 
and instruction. For example, parent involvement and support from state and local 
educational agencies were limited. Needs for more resources were identified. These 
included having sufficient faculty and staff, meeting the requirements of special needs 
students, and technological resources. Observation data suggest that few higher-order 
learning strategies and moderate levels of student-centered strategies were employed 
in the classrooms. 
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Cohort 2 



City University School of the Liberal Arts. When compared to last year and the 
available data sources, more accurate designations for levels of implementation were 
apparent this year. As indicated by the leadership team, most benchmarks seemed to 
be appropriately assigned to Phase II, with the exception of instruction. Traditional, 
teacher-directed instruction was dominant with little observational evidence to support 
the use of student-centered activities, higher-order strategies or the integration of 
technology. In regard to assessment, some of the suggested improvements in the 
benchmark document itself were not made. This is an essential tool in measuring 
implementation progress. As noted last year, some of the indicators are difficult to 
measure or objectively assess. For example, the indicator for Phase III under external 
support and assistance states that, “all decisions made by personnel at Cityll are in the 
best interest of our scholars and programs.” The evidence for this indicator includes list 
of recommendations and feedback from the president, professors, parents, and 
scholars. Thus, some benchmark indicators might be revised and better aligned with 
more objective evidence and written with more distinct gradations across Phases, 
culminating in more challenging long-term goals. To their credit, the leadership team 
has included benchmarks specifically addressing evaluation and identified appropriate 
indicators and evidence. Some of the evaluation benchmarks might be integrated as 
sources of evidence for other benchmarks throughout the document. For example, 
school climate surveys are listed as source of evidence for the benchmark related to 
evaluation strategies, but these data could also be used to inform benchmarks related 
to support or parent/ community involvement. The evaluation of assessment progress 
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was also confirmed by teachers’ endorsement of the statement that academic and non- 
academic goals are regularly reviewed by school staff. 

Star Academy. With the exception the implementation of new programs, the 
consensus of the leadership team was that the implementation of most goals and 
strategies were in Phase II of development. With regard to curriculum, the specific 
program identified for math and reading was McMillan McGraw Hill, and these programs 
do appear to be fully implemented. New programs included Voyager for reading and a 
character development program. Understandably, these programs are in earlier stages 
of implementation. Progress was apparent in the implementation of benchmarks 
targeting instruction. Cooperative learning and peer tutoring were aligned with their 
instructional strategies and were frequently observed during observations. Other 
student-centered and higher-order strategies were more commonly employed. 
However, they were not observed a majority of the time as prescribed in the final phase 
of their corresponding benchmark. (That goal, however, might be unrealistic and need 
to be moderated.) The indicators for benchmarks related to organization and support 
were considered to be in Phase II, and survey data support this ranking. Teachers 
responded positively about their involvement in decision-making and various areas of 
support and resources. The exception was low levels of agreement to the items about 
assistance from educational agencies or external partners, the adequacy of 
technological resources, and meeting the requirements of special needs students. As 
also noted last year, goals related to evaluation could be improved by revising the 
document to include benchmarks related to evaluation, assessment, and accountability 



162 




as well as more benchmarks in the category of support. All teachers did agree that 
goals were regularly reviewed to evaluate progress. 

Yo! Academy. Results suggest that Yo! has made substantial progress in the 
implementation of their benchmarks. In their benchmark document, levels of 
implementation were primarily identified as in Phase II. The exceptions were in the 
benchmarks targeting curriculum and instruction, which were designated as Phase III. 
The evaluation findings support these designations with the exception of the 
instructional category. Although observational data showed an increase in the use of 
higher-order strategies, there was little use of subject area integration and no project- 
based learning, which would be expected given the focus of an art-infused, integrated 
curriculum. Traditional pedagogical strategies were most frequently observed. And as 
noted last year, the benchmark document was not very specific about what curricula or 
instructional strategies were targeted. Evaluation efforts would be improved by more 
thoroughly detailing the curriculum aligned with instructional strategies. Benchmarks 
related to support and organization appeared to approach fuller implementation as 
evidenced by parent, teacher, and school climate data. 

Cohort 3 

KIPP Academy Nashville. Given that KIPP was in its initial year of operation, one 
would expect the implementation of benchmarks to be in early phases of development. 
The leadership team has judged that all benchmarks are in Phase I of development, 
and these rankings are supported by the evaluation findings. In the area of instructional 
strategies, their benchmark and indicators emphasized cooperative or collaborative 
learning. Based on observational findings, KIPP was judged to be strong in terms of 
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employing higher-order strategies, not in cooperative or collaborative learning. They 
might build on their pedagogical strengths and include benchmarks and evidence 
related to higher order learning strategies in conjunction with group learning 
orientations. The integration of technology in teaching is an area in need of 
improvement and they might also target this area in their benchmark document. These 
suggestions exemplify ways to improve the benchmark document to promote better 
planning and evaluation. Although KIPP is in the initial stages of implementation, the 
school has made progress in many areas that should advance it to the next stages of 
development. 

Memphis Business Academy. As would be expected for a school in its initial 
year of operation, most benchmarks are considered to be in Phase I and perhaps in 
early stages of Phase II for curriculum development and implementation. During 
interviews, the principal clearly described the infusion of business concepts into the 
curriculum and teachers described the adoption of specific curricular programs across 
disciplines. Although curriculum implementation might be considered to be in Phase II, 
benchmarks for instruction were more clearly in Phase I. If a variety of innovative, 
research-based strategies were consistently employed (as indicated), it would be 
expected that more student-centered strategies and more integration of technology 
would have been observed. The use of technology in a business-themed school would 
seem to be essential. The data indicate that neither professional development nor 
parental involvement emerged as strengths of this charter school and should be more 
appropriately considered to be in the initial phases of development. There were no 
indicators provided for parent and community involvement, which hinders evaluation 
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efforts in these areas. The leadership’s designation of Phase I in other benchmark 
categories was appropriate, even though the indicators and corresponding evidence 
could be improved. 

Promise Academy. The leadership team at Promise Academy has designated its 
school to be at Phase II for all but one benchmark. However, the evaluation findings 
suggest that Promise is in earlier stages of development for benchmarks related to 
curriculum, instruction, and evaluation. As the benchmarks are written, they may well 
be in the early stage of Phase II for support and organization. With regard to 
curriculum, several programs and associated strategies were identified during 
interviews, but teachers commented that they were just beginning to use the Socratic 
method, a strategy emphasized in their professional development activities. And, 
although they described the other curricular programs favorably, they were dissatisfied 
with Everyday Math and talked of pursuing a new math curriculum the following year. 
SOM and RSCA data indicate teacher directed instruction to be more prevalent, 
although student-centered and higher-order strategies were sometimes observed. It 
would appear that “teachers are beginning to use innovative, research-based strategies” 
as described in Phase I of their benchmark document. The leadership did recognize 
that Promise was in the Phase I with respect to the benchmark on measurable goals 
and standards. It seems that having measurable goals and standard go hand in hand 
with more advanced evaluation strategies. There is evidence for strong internal support 
but evidence for external support was more mixed. The types of evidence used to 
confirm indicators were weaker in this area. Specifically, the evidence across phases 
was simply listed as “support from various agencies.” 
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Southern Avenue Charter School of Academic Excellence. Southern Avenue 



Charter School of Academic Excellence seems to have made significant progress 
implementing its benchmarks in its initial year of operation. However, it is doubtful that 
all benchmarks are in the second and third phases of implementation as designated on 
Southern’s benchmark document. This discrepancy may be partially explained by the 
benchmarks themselves. For example, the benchmark under the umbrella of curriculum 
indicates that all strategies and methods are scientifically based; however, there is no 
indicator to gauge the quality or frequency of their use. A similar pattern was noted for 
other benchmarks. More specific indicators and evidence could be used to judge 
achievement in the three phases of development. The principal and teachers 
characterize their curricular programs as “progressing” and note some limitations in their 
science, math, and language arts programs. The benchmark document also designates 
their instructional strategies as in third and final phase of development. The 
observational data, however, show that direct and student-centered instruction was 
strong, but higher-order strategies and technology integration were weaker. Also, the 
findings do not support Phase III indicators that characterize instruction as fully 
differentiated and used by all teachers. Data do suggest more intermediate phases of 
development for the categories of support and organization. Teacher, parent, and 
school climate data suggest a good deal of internal support and organizational strength. 
Evidence for external support and parental involvement was more mixed. With respect 
to the evaluation category, some of the benchmarks targeting curriculum could be 
improved to better support the school evaluation and assessment. 
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Soulsville Charter School. Overall findings suggest that Soulsville Charter 
School is in the initial phases of development, as seems appropriate for its first year of 
operation. The leadership team also indicated they were at the initial stage of 
development for most benchmarks. However, there were some exceptions. In the area 
of curriculum, the principal stated the math curriculum, Math Renaissance, has not yet 
been implemented. They plan to implement this program next year (2006-2007). 
Further, teachers described some adjustments that may be needed in reading and 
science curricula to make them more appropriate for students in their grade levels. 
Their benchmark document also indicated, “Teachers routinely and skillfully use 
research-based teacher teaching strategies the majority of the time in all classes” 
(Phase III). This was not evidenced in the SOM and RSCA data that showed 
predominant use of direct instruction and rare use of student-centered strategies. 
School climate and teacher and principal survey data further support the findings that 
benchmarks under the categories of support, organization, and evaluation are in their 
initial phases of development. When compared to national norms, school climate data 
showed lower than average ratings on the dimensions of collaboration, environment, 
involvement, and leadership. More preliminary stages of development were also 
suggested by the teacher survey results indicating the need for more common planning 
time, more faculty and staff, and the ability to meet the requirements of special needs 
students. In addition, only 25% of teachers agreed that school staff regularly reviewed 
academic and non-academic goals to evaluate progress. 

Summary findings across schools. Table 8 provides the summary ratings for the 
level of implementation of goals and strategies in five broad areas. Not surprisingly, the 
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most advanced levels of implementation were observed among first cohort schools in 
their third year of operation. First cohort schools were either in intermediate or final 
phases of development across categories. The only exception was weaker levels of 
implementation of support systems at SCA, which included the need for more 
resources. Ratings for the implementation of goals and strategies tended to be 
moderate or intermediate for 2 nd cohort schools. Exceptions were lower levels of 
implementation in the categories of instruction (City U and Yo!) and evaluation (City U). 
Low to intermediate levels of implementation were apparent among 3 rd cohort schools in 
their initial year of operation. Somewhat more implementation progress was observed 
at Southern and Promise, with both rated as moderate in two or more categories. It 
should be noted that ratings are at least partially affected by the benchmarks 
themselves, with some schools setting the bar higher than others in terms of indicators 
and evidence. 

Overall, it was predictable that these charter schools have not fully implemented 
all benchmarks because it is a multi-year process compounded with the challenge of 
launching a new school. Continued improvement is implicit in the schools’ development 
goals. Across all schools, the strongest levels of implementation tended to be for 
benchmarks targeting support and organization. The achievement of these benchmarks 
may be a precursor to achieving full implementation of the curriculum and instructional 
strategies routinely employed at the individual classroom level. 
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Table 8 

Summary Ratings for Implementation of Goals and Strategies 



School Curriculum Instruction Support Organization Evaluation 




Beginning 



Moderate 




Strong 
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Question 5. What are teacher reactions to and experiences in the charter school? 
What are the adequacy and quality of professional development and resources? 

The Charter School Teacher Questionnaire (CSTQ) and teacher focus group 
data were used to address teachers’ reactions to and experiences in the charter school, 
including their responses targeting professional development and resources. The 
responses were organized into four categories. The first category (Program/Mission) 
encompassed two broad areas: teachers’ understanding of the mission and support of 
the educational program, and their perceptions that the program would result in 
successful student outcomes. The second category was professional development. 
The third category, resources, included whether teachers had adequate planning time, 
educational materials, technology, and if the school had a sufficient number of faculty 
and staff. The fourth and final category was support from the community, parents, and 
state and local educational agencies. See Table 9 for summary findings. 

Cohort 1 

Circles of Success Learning Academy. There seems little doubt that teachers at 
COSLA have continued to support their educational mission and program. There was 
unanimous agreement on all CSTQ items related to this category, and the ratings have 
been consistently high across the school’s three years of operation. For example, all 
teachers agreed that they have a thorough understanding of the school’s mission and 
the educational program is positively impacting student achievement. Teachers also 
perceived professional development as adequate and effective. Nearly all (83%) 
agreed that they received adequate professional development, and the level of 
agreement has remained high across years. In their focus group, teachers 
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characterized professional development as positive and cited conferences and 
mentoring sessions for new teachers offered by the district as examples. With regard to 
resources, teachers unanimously agreed that educational materials were readily 
available, but tended to disagree that they were given sufficient planning time or that the 
school had a sufficient number of faculty and staff (33% agreement on both). Open- 
ended comments identified the need for more support staff that included teaching 
assistants, a curriculum coordinator, and a guidance counselor. They also said they 
needed more time and partners for collaboration. As in previous years, CSTQ ratings 
indicate high parent participation. However, decreases in ratings on items related to 
support from other institutions and agencies were noted. None agreed that they 
received effective assistance from the school district, and only one-third of respondents 
rated the assistance received from the State Department of Education as effective. 
During the focus group, the teachers remarked that Memphis City School teachers and 
employees were not supportive of charter schools. 

Memphis Academy of Health Sciences. Both quantitative ratings and qualitative 
responses reflected strong teacher support for the educational mission and program at 
MAHS as well as the belief that they are positively impacting student success. These 
findings have been apparent in all three years. In contrast, professional development 
opportunities to implement the educational program have been viewed less favorably. 
Nearly two-thirds (60%) of the teachers agreed that professional development has been 
adequate, while 40% were either neutral or disagreed. Teachers commented on the 
need for more professional development in open-ended responses to the CSTQ and in 
the focus group. With regard to resources, the findings were somewhat mixed. Most 
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teachers concurred (80%) that the school had sufficient numbers of faculty and staff, but 
tended to disagree that they had adequate technological resources (80% disagreed or 
were neutral) or educational materials (60% disagreed or were neutral). One-third of 
respondents further disagreed that they were given sufficient planning time. The need 
for more technology and planning time also emerged in the qualitative responses. 
Although parent participation and community support were considered to be strong, 
support from the state and local educational agencies was perceived to be lacking. 
Only 20% of teachers agreed that they received effective assistance from these 
agencies, which were similar to the levels of agreement observed in previous years. 
One comment specifically identified “more district-wide support and encouragement” as 
an area in need of improvement. 

Memphis Academy of Science and Engineering. Teachers’ reactions to their 
experiences in MASE were extremely positive, especially when compared to previous 
years. The increasingly high levels of agreement on CSTQ items were apparent across 
the categories of support for the educational program, professional development, 
resources, and support from educational agencies, external partners, the community, 
and parents. The only exception was the 50% level of agreement that the program 
adequately meets the requirement of children with special needs. However, this 
percentage increased from only 10% agreement for this item last year. Teachers’ open- 
ended comments were largely positive and support the item responses. However, 
when asked what they consider negative aspects of their charter school or areas in 
need of improvement, communication problems among staff, parents, and 
administrators were identified. Some of the communication difficulties seemed to relate 
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to the lack of advance planning and notification about events. Another recurrent 
response was the lack of space in the school. 

Smithson-Craighead Academy. Teachers’ reactions to and experiences at SCA 
have improved with time. The CSTQ findings show unanimous or nearly unanimous 
support for the educational program, understanding of the school mission, and 
expectations of student success at this charter school. Levels of agreement on whether 
teachers are adequately involved in decision-making increased markedly across years. 
Responses to this item increased from 33% and 20% agreement in the first two years, 
respectively, to over 90% agreement in the third year. There has also been increasingly 
strong endorsement for the adequacy of professional development to implement the 
educational program (from 50% in the first year to over 90% in the third year). 

Teachers’ open-ended responses indicated that they received in-service training 
at the school, and few resources for professional development were provided by the 
district. This type of comment provides some explanation for the lower ratings observed 
on some items related to support for the charter school. Specifically, over 70% of 
respondents disagreed or were neutral on whether the school receives effective 
assistance from state and local educational agencies. A higher percentage of teachers 
believed that the community’s perception of the charter school was positive (73%), that 
the school receives effective assistance from external partners (73%), and that parents 
are offered substantial opportunities to participate in their child’s education (91%). 
Rating gains were observed on all three of these items across years. Even though 
respondents believed that parents had opportunity to participate in their child’s 
education, they did not consider parents to be active partners with the school (64% 
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either disagreed or were neutral on this item). Open-ended responses were also 
somewhat mixed with regard to perceptions about parental involvement. For instance 
teachers noted that parents “had great things to say about the school,” whereas another 
suggested the need for parent training to increase their support, particularly in the area 
of student conduct and discipline. Teachers endorsed most statements associated with 
the adequacy of resources. The only exception targeted sufficient numbers of faculty 
and staff. Just over 50% of teachers agreed with this statement. 

Cohort 2 

City University School of Liberal Arts. Survey findings clearly reflect teachers’ 
support of the educational program at City U and a thorough understanding of its 
mission. All teachers indicated they are involved in the decision-making at this school. 
The high percentages of teachers endorsing these characteristics were similar to last 
year’s findings. Although respondents believed that City U promoted student 
achievement, a lower percentage of respondents (67%) agreed that the educational 
program encouraged students to have high standards for their own work when 
contrasted to last year (88%). Open-ended responses did not illuminate this change in 
perceptions. 

Compared to last year’s ratings, more teachers agreed that they received 
adequate professional development (83% in the second year and 75% in the first). 
Focus group responses revealed that teachers had training in the areas of on-line 
grading, classroom management, and special education as well as discussion of a book 
on quality schools. Professional development was described as “informative and 
effective.” The pattern of results pertaining to resources, however, was somewhat 
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mixed. In the areas of technology, educational materials, and addressing the 
requirements of special needs students, the findings were more positive and 
demonstrate an increase from the previous years. However, there was less 

endorsement of the assertion that there was sufficient faculty and staff (33% agreement 
in the second year versus 88% agreement in the first year). Open-ended comments 
reinforced the need for more staff to teach resource classes. Although support from 
parents and community were considered to be strong, support from the local 
educational agency and external partners was not apparent. Only 17% of the teachers 
agreed that they received effective assistance from the school district, and only one- 
third agreed that they received effective assistance from external partners. 

Star Academy. There was unanimous agreement among teachers on CSTQ 
items that reflected support for and understanding of the school’s educational program, 
the promotion of student achievement, and the adequacy of professional development. 
These impressive results were also observed in the previous year and supported by 
qualitative findings. Notable decreases in the ratings were apparent among some items 
pertaining to the adequacy and availability of resources across years. Fewer teachers 
agreed that there was a sufficient number of faculty and staff members (64% in the 
second year compared to 100% in the first), that they had adequate technological 
resources (27% versus 86%), or that the requirements of special needs students were 
adequately addressed (27% versus 86%). In their comments, teachers cited the need 
for more technology, more classroom space, and more assistance with special 
education. However, teacher responses did indicate that planning time was sufficient 
(82% agreement) and educational materials were readily available (91% agreement). 
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With regard to support, perceptions of parent involvement and participation remained 
high, whereas support from state or local educational agencies and external partners 
was viewed less favorably in both an absolute sense and relative to the previous year. 
The percentage of agreement was only 27% in these three areas. The lack of support 
from the state, district or external agencies did not emerge as a theme in the qualitative 
findings. 

Yo! Academy. Compared to the first year of operation, questionnaire data show 
that teachers’ reactions and experiences have improved in all areas. The percent of 
agreement was unanimous, or nearly so, for all items related to support for the 
educational program and high expectations for student success. Perceptions of 
professional development were more positive in this second year (from 56% to 89% 
agreement). During focus groups, faculty described a wide array of professional 
development activities including training in the use of a technological tool to analyze 
data, discussions on classroom management and rapport, and graduate coursework. 
Ratings on items related to resources were also high and showed improvement 
compared to the previous year. The lowest level of agreement was observed on the 
item assessing the availability of educational materials (67% compared to 33% in the 
first year). The largest gain in scores was obtained on the item assessing the adequacy 
of technological resources (89% compared to 33% in the first year). However, one 
called for expansion of technological training, while two others called for more resources 
in art and classroom materials. The CSTQ findings on items that reflected support were 
mixed. All teachers agreed that parent participation was strong, and most concurred 
that community perceptions of Yo! were positive (78%). Qualitative responses about 
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community support and perceptions were also positive. In contrast, the levels of 
agreement for items reflecting support from local and state agencies were much lower 
(44% and 56%, respectively). The teachers did describe resources provided by the 
school district during the focus group interview but did not address the effectiveness of 
assistance. 

Cohort 3 

KIPP Academy Nashville. Teacher responses to the CSTQ reflected strong 
support and understanding of the school’s educational program and mission. There 
was 100% agreement on items related to this category. They also believed that the 
educational program was positively impacting student achievement and encouraging 
high academic standards. Their reactions to the adequacy of professional development 
were also positive (80%). In the interview, teachers described their professional 
development activities favorably. They appreciated the training tailored to charter 
schools and the latitude to focus development efforts in areas they thought they 
needed. Results pertaining to the availability and adequacy of resources were mixed. 
Less than half of the teachers agreed that they were given sufficient planning time, the 
school had a sufficient number of faculty and staff members, or had adequate 
technological resources. One comment targeted the lack of materials and resources as 
an area in need of improvement: “They have acquired lots of things this year, but this is 
a first-year school and materials are still limited.” According to teachers, support from 
the state and school district were lacking. No one agreed that these educational 
agencies provided effective assistance. Teachers were much more positive about 
external partners, parents, and the community. 
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Memphis Business Academy. All four respondents uniformly endorsed items 
related to support and understanding of the educational program and had high 
expectations for the student academic success. Only half agreed that they adequate 
professional development to support their program. One teacher commented that more 
mentoring of first-year teachers was needed. Findings pertaining to resources varied. 
There was strong agreement that technological resources, teacher planning time, and 
the ability to meet the requirements of special needs students were adequate. Lower 
levels of agreement were found for having sufficient faculty and staff, and availability of 
educational materials. External support was considered weak in most areas. No 
respondents agreed that they received effective assistance from state or local 
educational agencies or that parents were active partners in the schools. In contrast, 
they all agreed that they received effective assistance from external partners, and that 
parents were offered substantial opportunities to participate in their child’s education. 
While 100% of teachers agreed parents had substantial opportunities for involvement, 
75% disagreed that parents were active partners in the school. In open-ended 
comments, the lack of parental involvement was repeatedly cited as an area in need of 
improvement. Three of the four teachers considered community perceptions of the 
school to be positive. 

Promise Academy. The three teachers who responded to the CSTQ agreed or 
strongly agreed to nearly all items across areas. There was 100% agreement on 16 of 
the 20 items, indicating very favorable reactions to the educational program, 
expectations for student success, professional development, resources, and parental 
support. The only exception to this pattern of uniformly high ratings was on items 
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reflecting assistance from state and local educational agencies (33% agreement), 
external partners, and community support (67% agreement for both items). The 
qualitative responses were primarily favorable. Teachers expanded upon their ratings 
by identifying resources and support for their educational mission, particularly in relation 
to having high expectations for student success. One said, “They are given 
opportunities to explore, experiment, investigate and be involved in educational 
opportunities that can use throughout life.” The lower rating related to support from the 
local educational agency may have been due to a perceived threat to autonomy. When 
asked what they considered a negative aspect of their charter school, one teacher said 
it was the inability to “change aspects of the program without having to go through LEA 
board.” 

Southern Avenue Charter School of Academic Excellence. Teachers indicated 
strong support for Southern’s educational program including its high expectations for the 
academic accomplishments of students. All four of the respondents agreed to these 
items. Three teachers also agreed that they received adequate professional 
development. In the focus group, they favorably described their professional 
development activities, which focused on classroom management and instructional 
“best practices” that they could “easily” apply in their classrooms. Questionnaire ratings 
related to resources were largely favorable. All teachers considered educational 
materials as readily available, technological resources as adequate, and the number of 
faculty and staff as sufficient. However, only half of the teachers thought they had 
sufficient planning time, and only one (25%) agreed that the educational program 
adequately addressed the requirements of special needs students. Two of the open- 
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ended comments identified lack of consistent planning time to be a negative aspect of 
their school. Only half of the teachers agreed that they received effective assistance 
from local educational agencies, the state, or external partners. All considered parents 
to be involved with substantial opportunities for participation and perceived community 
reactions to be positive (100% agreement). 

Soulsville Charter School. Of the four teachers who responded to the 
questionnaire, most agreed with items indicative of support for the educational program 
(100%), understanding of the school mission (75%), and beliefs that the program was 
positively impacting student achievement (75% across 3 items). Apparently one of the 
four teachers did not respond to these last items resulting in 75% percent agreement 
rather than 100%. Only two of the teachers perceived professional development to be 
adequate. Teachers commented that more thorough professional development was 
needed before the implementation of new programs. Both quantitative and qualitative 
responses suggested that resources such as common planning time, sufficient faculty 
and staff, and the ability to meet the requirements of special needs students were 
lacking. In open-ended responses, teachers commented that the school days were too 
long and often included Saturdays, that they had little time for planning, and that there 
were too few staff members with no substitutes or tutors. Three of the teachers agreed 
educational materials and technological resources were available. One teacher said 
the school had more resources due to extra funding that enabled them to implement 
programs that would not be in the budget otherwise. The findings reflect low levels of 
external support, especially from parents. One teacher remarked that parental support 
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was available, yet they did not take full advantage of this resource. On the other hand, 
three teachers agreed that community support was positive. 

Summary findings across schools. Overall, teachers’ reactions to and 
experiences in the charter schools tended to be very positive. Responses related to 
support of the educational program, understanding of the mission, and the likelihood for 
student success were particularly favorable. All charter schools were considered to be 
strong in this category. Most charter schools were also rated as strong in the area of 
professional development. Only two charter schools (MAHS and MBA) were judged to 
be moderately successful in this area, and one first-year charter school (Soulsville) was 
evaluated as low or weak. Teacher perceptions of the availability and adequacy of 
resources were more varied with most schools rated as moderate. Three schools 
(MASE, Yo! and Promise) were rated as strong in this category and one (MAHS) was 
rated low. Although there were differences by school in the type of resources that 
respondents endorsed as sufficient or adequate, having enough common planning time 
and faculty or staff more commonly emerged as areas in need of improvement. With 
regard to perceptions of support or assistance from educational agencies, the 
community, and parents, teachers’ ratings and comments were less favorable. Only 
one school (MASE) was judged as strong in these three areas of support. In all other 
schools, teachers consistently reported a lack of effective assistance provided by state 
and local educational agencies, and this finding may be more of a reflection on these 
agencies rather than the charter schools. Two schools (MBA and Soulsville) were rated 
as low in the support category, and both schools were in their initial year of operation. 
In terms of trends observed across years, there was notable improvement in teachers’ 
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reactions in many schools. Given that the perceptions were largely positive in previous 
years, this continued improvement is impressive. However, findings should be 
interpreted with some caution because the sizes of the faculty tended to be small, 
particularly in Cohort 3 schools. With very small samples, anonymity and candor may 
be threatened. 
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Table 9 

Summary Ratings for Teachers’ Reactions 



School Program' Professional Resources support 
mission development 




Low/Weak 



Moderate 



Strong 
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Question 6. What are parent (caregiver) reactions to and experiences with the charter 
school? 

To address this final evaluation question, we relied primarily on quantitative and 
qualitative data collected on the parent questionnaire. It should be noted, however, that 
the return rate for parent questionnaires at some of the schools was low, as normally 
occurs in schools serving low socioeconomic areas; thus the sample data may not be 
generalizable to the parent populations at the respective schools. For example, when 
we divide the number of completed parent questionnaires by the number of students 
enrolled, the response rate at Cityll was only 15% and only 13% at MASE. In other 
schools the response rate was about 50%. The parent questionnaire results were 
augmented by the teacher questionnaire, teacher focus group, and the principal 
interview. For ease of expression, we refer to “parents,” but our use of this term also 
encompasses caregivers and guardians. The findings are organized into two 
categories: parent satisfaction and parent involvement. 

Cohort 1 

Circles of Success Learning Academy. Parent satisfaction ratings and 
comments remained positive across years on most questionnaire items, but as noted 
earlier the response rate was only 15% at this charter school for the most current year. 
At least 80% of the respondents agreed with 14 of the 17 positive attributes of the 
charter school, and these high ratings were consistent across years. The highest level 
of agreement was observed on items assessing the quality of instruction provided by 
teachers and the encouragement of parental involvement (97% for both items). In their 
comments, many parents praised the small class sizes that contributed to high-quality 
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instruction. Notable declines in ratings were observed on two items related to the 
principal’s leadership (from 98% agreement in the second year to 57% in the third year) 
and the attractiveness of classrooms and hallways (from 98% agreement in the second 
year to 71% in the third year). The decrease in ratings pertaining to principal leadership 
may be due to the fact that there is a new principal at COSLA this year. In their open- 
ended responses, parents identified the need for more physical space for classes, 
cafeteria, auditorium, play areas; better transportation and parking options; more 
nutritious, better quality food; and more extracurricular activities. 

Data obtained from parent and teacher questionnaires point to a high level of 
parent involvement at COSLA. Many parents commented that their involvement was a 
strength of this charter school. Teachers unanimously agreed with the questionnaire 
item stating that parents were active partners with the school. In open-ended 
comments, teachers cited parent involvement and participation as one of the most 
successful aspects of their charter school. The interview responses are somewhat 
contradictory because teachers and the principal both noted a decrease in parent 
involvement. However, teachers went on to explain that they still considered parent 
involvement to be strong but attribute the decrease to the change in administrators. 
They also suggested that parents felt more comfortable with the school now that the 
“newness has worn off” and don’t need to continually monitor activities. Even though 
there was some variation noted across years, the formal contract signed by parents to 
volunteer for at least 20 hours may account for the continued higher levels of parent 
involvement in this school. 
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Memphis Academy of Health Sciences. Although the majority of parent 
satisfaction ratings remained high, there was some decline in scores when compared to 
the previous year. For most items, the drop in ratings was small. Some of this variation 
may be due to the low response rate this year compared to last. Whereas 86 parents 
responded last year, only 29 responded this year. Parents seemed least satisfied with 
items reflecting the physical environment (classrooms, hallways, school grounds, and 
transportation). Additionally, there were notable decreases in the percentage of 
agreement on teacher availability and whether their child liked attending the school 
(79% agreement for both items). When compared to last year, many of the same 
strengths emerged in the open-ended comments and support the rating scale results. 
Parents cited the academically challenging curriculum, the small student-to-teacher 
ratios, high expectations of students, principal leadership, and teacher quality. With 
only a couple of exceptions, praise for order and discipline at MAHS emerged as an 
additional theme in the open-ended comments. In response to suggestions for 
improvement, parents still hoped to have a new building and some of the advantages a 
new building would afford, such as a library, school office, cafeteria, and more space in 
general. Other suggestions were to have a guidance counselor and to add more grade 
levels and specialized classes (e.g. languages, health). 

The evidence pertaining to parental involvement was mixed. In open-ended 
comments, some parents noted the lack of parent involvement and suggested the need 
for improved communication with school staff. Teacher questionnaire data revealed an 
increase in the percentage of teachers who agreed that parents are active participants 
in the school (from 36% percent in the second year to 73% in the third). In contrast, all 
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teachers agreed that the school offered parents substantial opportunities to participate 
in the education of their children. In their open-ended comments, one teacher 
considered parent involvement to be exceptional, yet another thought it was high but 
could be improved. During interviews, the principal described parent involvement as 
constant, while teachers described it as increasing. Some of the variation in responses 
may be due to how parental involvement is defined and then evaluated. This is 
reflected in the following comment from the teacher focus group. “The school is the 
parent’s community, but they are not educated on how to support us.” 

Memphis Academy of Science and Engineering. With few exceptions, parents 
remained very satisfied with the education provided at MASE. However, the response 
rate of 13% was particularly low at this school. The rating scale data indicate 
consistently high rates of endorsement in nearly all areas across years. The three 
exceptions were the relatively lower levels of agreement on items related to the 
attractiveness of schools and hallways (66% agreement), the adequacy of 
transportation (42%), and regular receipt of information about their child’s progress 
(59%). These items also received lower levels of endorsement in previous years. The 
qualitative data was primarily favorable. School strengths identified in parents’ open- 
ended responses were similar to those in previous years. Most parents cited the 
challenging curriculum, leadership, and dedicated teachers as strengths of the school. 
Other themes included the high expectations for students, school safety, discipline, and 
individual attention or caring for students. In reference to areas in need of 
improvement, parents disliked the long hours, transportation problems, and the lack of 
timely communication and interaction with staff. They suggested more traffic guidance 
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for pick-ups and drop-offs, greater use of technology, and more extracurricular activities, 
particularly athletics or physical education. 

Evidence for levels of parental involvement was mixed. As noted, parents 
identified lack of communication with teachers and staff as limitations. They also 
mentioned the need for interaction with parents and timely communication and feedback 
about their child and school events. In contrast, teachers had very positive perceptions 
of parent involvement. On rating scale items, all teachers agreed that parents were 
active partners in the school and were provided substantial opportunities for 
participation. In open-ended comments, they cited parental involvement as strengths of 
their charter school. Interview data from the principal and teachers also suggest strong 
levels of parental involvement. Again, different types of involvement may be perceived 
and valued differently by parents and staff. Many parents may want more timely, 
consistent communication about their child’s educational progress and events even 
though they may be involved in fund-raising, committees, and other volunteer activities. 

Smithson-Craighead Academy. Both quantitative and qualitative questionnaire 
data indicate very high levels of parent satisfaction at SCA. The rating scale results 
show that levels of parent satisfaction have remained consistently high across items 
and years. Percentages of agreement were nearly 90% or above for all but two items. 
The first was challenge of the curriculum, which showed a decline from 85% in the 
second year to 73% in the third. And 74% of the parents agreed that the classrooms 
and hallways are attractive (compared to 80% last year). The dominant theme to 
emerge from parent comments about the school’s strengths was their appreciation of 
the teachers. Teachers were described as caring, helpful, concerned, and skilled. 
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Other strengths noted were smaller class sizes, quality curricula, and a safe, nurturing 
yet structured climate. Responses to areas in need of improvement were varied. Many 
parents said “nothing” needed improvement, and that they were very satisfied with the 
school. Other recurring suggestions were related to the physical space (e.g., having a 
library or playground), transportation, and extending the number of grade levels served. 

In contrast to the high levels of parent satisfaction, teacher data suggested that 
parental involvement was not as strong. Only 36% of the teachers agreed that parents 
were active partners in the school. By comparison, 91% agreed that parents were 
offered substantial opportunities to participate in the education of their children. An 
open-ended comment suggested that many parents did not model appropriate behavior 
for their children and suggested the need for parent training. In focus groups, the 
teachers noted that although parents are “saying great things about the school.” the 
support from the community was sometimes mixed. They went on to explain that SCA 
was not a neighborhood school for most of these students, which may account for less 
involvement from the local community. The principal stated that parents were pleased 
with the services provided at the school and were involved but was not specific in 
describing the ways in which parents were involved. Although there seemed to be 
some disagreement about the level of parent involvement, teachers and the principal 
favorably described the involvement of grandparents via a program offered from Metro 
City Services. 

Cohort 2 

City University School of Liberal Arts. Responses to the parent questionnaire 
revealed a good deal of variation in responses. There was a general decline in scores 
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compared to the previous year’s data, but the percentages of agreement remained high 
(over 79%) on 8 of the 17 items. The largest declines were observed on items 
assessing whether the school grounds were well maintained (from 82% to 41%), 
teachers were available to parents (from 100% to 64%), the curriculum was appropriate 
(from 100% to 73%), and regular information about their child’s progress was received 
(from 100% to 73%). The two items with relatively low agreement rates across years 
concerned the attractiveness of classrooms and hallways (41%), and the adequacy of 
transportation services (36%). In open-ended comments, parents cited as strengths the 
challenging academic program, the small class sizes that afforded individual student 
attention, and the nurturing relationships between students and staff. Recurring topics 
in parent suggestions were to include more extracurricular activities, provide more 
space, and promote a more open, respectful climate. Overall, most parent comments 
about City-U were very favorable. 

The evidence suggests high levels of parent involvement, which is especially 
impressive for a high school. The principal described parent involvement as increasing 
and “excellent.” In their focus group, teachers essentially made the same comments 
about the increasing involvement of parents. On the teacher questionnaire 100% 
agreed that parents were active partners with the school and noted parent involvement 
as a strength in their open-ended comments. 

Star Academy. Similar to last year, rating scale data indicated very high levels of 
satisfaction on parent questionnaire items. Nearly all percentages of agreement ranged 
from 95% to 100%. Adequacy of transportation services was the only item that received 
a lower level of agreement (65%). Parent open-ended comments were overwhelmingly 



190 




positive. Among the numerous strengths noted were a challenging curriculum, high 
expectations for students, low student-to-teacher ratios, teacher dedication, principal 
leadership, parental involvement, and a safe environment for their children. When 
asked what they would like to see improved at the school, many parents responded with 
“nothing” or “everything is wonderful.” Some recurring suggestions were to add more 
physical space that would afford larger classrooms, a cafeteria, a library, adding more 
grade levels, improving transportation, and offering athletic or other specialized 
programs. 

Evidence points to strong levels of parental involvement. Ratings on the teacher 
questionnaire showed about 82% agreement with the statement that parents are active 
partners with the school and 100% agreement on the item indicating that parents are 
offered substantial opportunities to participate in the education of their children. When 
teachers were asked about strengths of their school and how it differed from a regular 
public school, high levels of parental involvement emerged as a recurring topic. These 
favorable sentiments regarding parental involvement were also expressed during the 
teacher focus groups. The principal described parental support as on the rise and 
noted they were active participants in field trips, school productions, and monthly 
parent-teacher meetings. High levels of parent involvement may be attributable to the 
formal contract signed by parents that specifies they will volunteer a minimum of 20 
hours per year. 

Yo! Academy. Although the parent questionnaire suggested mostly high levels of 
satisfaction, a decline in ratings compared to previous year was observed. The largest 
declines were seen on items reflecting consistent enforcement of rules (from 96% to 
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67%), school safety (from 92% to 72%), principal leadership (from 92% to 72%), and 
the appropriateness of the curriculum (from 96% to 72%). Surprisingly, the parents’ 
open-ended comments did not shed light on possible reasons for the decrease in 
agreement levels. When parents were asked about areas in need of improvement, 
many commented with “nothing” or that they “love it just the way it is.” Identified 
strengths included the dedication and quality of teachers, smaller class sizes, and 
school leadership. 

In contrast to some decline in parent satisfaction ratings, other indicators suggest 
an increase in the level of parent involvement. All teachers (100%) agreed that parents 
were active partners in school and were offered substantial opportunities to participate 
in the education of their children. Teachers also noted that, “parental support was 
strong at the outset and has continued to grow.” The principal and teachers concurred 
that they have high attendance at parent-teacher meetings, but the principal estimated 
that only half of the parents were actively involved. According to the teachers, they 
have personally interacted with all parents, but the level of participation for some 
parents may not extend beyond communication and meeting attendance. 

Cohort 3 

KIPP Academy Nashville. Questionnaire data indicate that parents are very 
satisfied with nearly all aspects of their children’s education at KIPP. The percentages 
of agreement ranged from a low of 68% on the attractiveness of hallways and 
classrooms to 100% agreement on four items: high quality instruction, principal 

leadership, high expectations for students, and being pleased that their child attends 
KIPP. The only other item that had relatively low levels of agreement was related to the 
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maintenance of school grounds (77%). Parents’ qualitative data supported the 
quantitative rankings. They particularly appreciated the quality teachers, strong 
leadership, challenging curriculum, and high expectations of students. In addition to 
comments indicating no improvement was needed, other parents suggested more 
extracurricular activities, extension of grade levels into high school, and providing 
student lockers. 

Although parents seemed clearly satisfied with this charter school, their level of 
involvement was not as clear. On the questionnaire, four of the five teachers did agree 
that parents were active partners in the school, but one teacher specifically noted the 
lack of parent involvement as a negative aspect or area in need of improvement. In 
interviews, the teachers said involvement was consistently high. The principal noted 
high attendance at parent meetings and remarked that the school provided regular 
feedback to parents on their children’s progress. On the School Climate Inventory, the 
average rating on the Involvement dimension which primarily assessed parental 
participation was lower than the means on other dimensions. Given the mixed 
evidence, parent involvement was judged to be moderate at KIPP. 

Memphis Business Academy. Questionnaire data revealed high rates of parent 
satisfaction. With only one exception, the percentages of agreement ranged from 82% 
to 100%. The three items with 100% agreement reflected parent satisfaction with the 
high expectations for student achievement, the encouragement of parental involvement, 
and treating parents respectfully. The only exception was their apparent dissatisfaction 
with transportation services (48% agreement). The parents’ open-ended comments 
support the quantitative data. Small class sizes that afforded more individual attention 
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for students was most commonly cited, but other strengths included the high 
expectations for students and a strong, challenging curriculum. Parents commented 
that they appreciated the regular communication from teachers, respect from school 
personnel, and the “open-door policy” at MBA. Although the positive comments 
outweighed the negative, some parents mentioned disciplinary procedures as in need of 
improvement and also suggested more physical education activities. 

In contrast to parents’ high level of satisfaction with MBA, they do not appear to 
be actively involved in the school. None of the teachers agreed that parents were active 
partners in the school, even though they all agreed that parents were offered substantial 
opportunities to participate in the education of their children. The lack of parental 
involvement also emerged as a consistent theme when teachers were asked to identify 
negative aspects or areas in need of improvement at their school. During focus groups 
teachers commented that they had an “open door” for parents and welcomed their 
support, but parent support was low and in need of improvement. For increasing parent 
involvement, the principal described plans that included hiring a part-time parent 
coordinator and arranging mandatory parent meetings. 

Promise Academy. Based on questionnaire data, the parents appear to be very 
satisfied with all aspects of their children’s education at Promise. The levels of 
agreement ranged from a low of 83% to a high of 100%. The highest ratings (100% 
agreement) were observed on the three items assessing teachers’ provision of high 
quality instruction, school safety, and encouragement of parental involvement. The 
parent comments were overwhelming positive with praise for the care and nurturing 
their children receive at Promise. A familial, loving atmosphere combined with a strong 
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curriculum and dedicated teachers were recurrent themes. Many parents responded 
that there was “nothing” to improve, while others suggested more space and 
extracurricular activities. However, parents did note that parental involvement needed 
to be improved with specific mention of more involvement in PTO meetings. 

With the exception of lower participation at PTO meetings, the teachers and 
principal indicated that parents were involved in the school. All three of the teachers 
who responded to the questionnaire agreed that parents were active partners in the 
school and were offered substantial opportunities to participate in the education of their 
children. In teacher interviews, their comments concurred with those of parents 
indicating low or “varied” involvement in PTO, but they maintained that there was a 
large parent turnout at special events and regular communication with parents. The 
principal contended that parental involvement had increased since the school opened 
and parents were regularly kept informed. 

Southern Avenue Charter School of Academic Excellence. With rare exceptions, 
consistently high levels of parent satisfaction were apparent at Southern. Most levels of 
agreement on the questionnaire hovered close to 90% or above. All parents (100%) 
agreed that they regularly received information about their child’s progress, and that 
school personnel treated them respectfully. Parents seemed dissatisfied with the 
transportation services (36%), and only moderately satisfied (78%) that rules for student 
behavior were consistently enforced. Several parents identified their involvement as 
strengths of the school. They also appreciated the small class sizes, the challenging 
curriculum, and caring atmosphere for students. Although most parents identified 
parental involvement as a strength at Southern, a few others considered it an area in 
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need of improvement. Other parents identified discipline policies and transportation as 
areas in need of improvement. Some hoped that more grade levels would be added. 

Most parents, teachers, and the principal concurred that there were high levels of 
parent involvement. All four teachers respondents agreed that parents were active 
partners and offered substantial opportunities to participate in the education of their 
children. In the focus group, teachers stated that parent involvement has remained 
consistent since the school opened. They explained that parents are required to 
volunteer for at least 10 hours a semester and spend 30 minutes per day reading with 
their child. The principal characterized parental involvement as increasing and 
specifically described numerous efforts to inform and involve parents in the school. The 
formal parent engagement contract may account for higher levels of parent involvement 
at this charter school. 

Soulsville Charter School. Questionnaire data suggested primarily high levels of 
parent satisfaction. Most levels of agreement on survey items were 80% or above. 
The highest percentages were obtained on the maintenance of school grounds (100%), 
school safety (97%), and high expectations for student achievement (97%). More 
moderate levels of agreement were obtained on whether the principal was a good 
leader (74%) and the availability of teachers (77%). It makes sense that some parents 
appreciated the musically enriched program at Soulsville. Others appreciated the 
challenge of the curriculum as well as the order, discipline, and safety at Soulsville. 
Whereas many parents cited order and discipline as strengths of the schools, others 
found the policies too severe and in need of improvement. Other suggestions for 
improvement were the inclusion of more physical education and other extracurricular 
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activities. It is noteworthy that a recurrent theme was the need for more parental 
involvement and regular communication with the school personnel. 

Consistent with the parents’ perceptions, teachers also indicated low levels of 
parental involvement. None of the four teacher respondents agreed that parents were 
offered substantial opportunities to participate in education of their children, and only 
one teacher (25%) agreed that parents were active partners in the school. In open- 
ended questionnaire comments and in the focus group, the teachers described the 
principal’s difficulty in communicating with and encouraging the participation of parents. 
The principal, however, characterized parental support as good and stated that parental 
involvement has remained constant. 

Summary findings across schools. All 12 charter schools were rated as strong 
with respect to parent satisfaction (see Table 10). These findings were similar to those 
obtained in previous years. Clearly, the parents had primarily favorable reactions to 
these schools. Both parent rating scale data and their open-ended responses 
supported these findings. Although most responses were favorable, some common 
areas were identified as in need of improvement. These included more extracurricular 
activities, particularly physical education or athletics, better transportation or parking 
options, and more physical space for the school. As was true in previous years, ratings 
in the category of involvement were more varied. The findings suggest that strong 
levels of satisfaction do not necessarily correspond to high levels of involvement. There 
was a pattern of more moderate to low levels of involvement in third cohort schools, 
which would make sense for schools just launching their programs. A pattern of 
findings by grade level was also detected. Three of the four schools judged to have 
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high levels of parent involvement served early elementary grade levels. The exception 
was City-U, a secondary school in the second cohort. The charter schools rated as low 
in terms of parent involvement were third cohort schools serving sixth graders. The 
exception was SCA, a first cohort school serving Kindergarten through 2 nd grade 
students. 
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Table 10 

Summary Results for Parental Satisfaction and Involvement 



Satisfaction Involvement 




Beginning/Low 

□ 

Moderate 



Strong 
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Conclusions and Recommendations 



Conclusions by Evaluation Question 

The present study used a variety of data sources to address the questions 
guiding the third-year evaluation of Tennessee charter schools. There were a total of 
12 charter schools evaluated. The first cohort of four schools was initiated in the 2003- 
2004 school year and has completed its third year of operation. The second cohort of 
three schools began operating in the 2004-2005 school year and has completed its 
second year. The final and third cohort of five schools began in 2005-2006 and just 
completed their initial year. 

The first evaluation question focused on instructional orientation and strategies, 
and the findings were similar to those obtained in previous years. Traditional, direct 
instruction remained the dominant pedagogical practice across charter schools. The 
rates were comparable to national norms in many cases, but still reflect limited success 
by the schools to implement the more innovative pedagogy described in benchmarks 
and instructional plans, or to take advantage of some of the unique conditions afforded 
by their charter status. One area in need of improvement was the integration of 
technology into classroom instruction. In some schools, the lack of technology 
integration may be due to a lack of resources, but in others it may be due to restricting 
its use to lower-level tasks (e.g., typing, calculations, reading) rather than to enrich and 
advance higher-level learning. RSCA data show that even when technology was 
employed in classrooms, its application to support higher-order thinking was limited. 
Commonly cited features of the charter schools were smaller class sizes and more 
challenging instruction and curriculum. One might expect therefore to observe more 
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student-centered, higher-order strategies supported by technology implemented in 
these instructional environments. This expectation was reinforced by the schools’ own 
benchmarks comprising their school improvement plans in the areas of curriculum and 
instruction. Although there were notable increases in these types of strategies among 
some schools, the rates were still low to moderate overall. 

School climate was addressed by the next research question. As reported last 
year, a favorable school climate continues to be an impressive strength of these charter 
schools. In some schools, the findings showed increases from already high averages 
when compared to previous years, and in second cohort schools, the decline in scores 
observed in the previous year appeared to rebound. On all dimensions, the SCI ratings 
obtained for the charter schools were higher than national norms. Still, as true in the 
national sample, scores on Order tended to be lower than those on other dimensions. 
Two third cohort schools, MBA and Soulsville, were designated as having lower indices 
of school climate, primarily due to problems with order and disciple. Consistent with 
findings in previous years, order and discipline problems also emerged in qualitative 
data collected at SCA. Nevertheless, less favorable evaluation of school climate was 
the exception across charter schools. 

The next question addressed the degree and level of quality with which the goals 
and strategies of the charter school were implemented. Not unexpectedly, the levels of 
benchmark implementation were related to cohort or year of operation. First cohort 
schools were either in intermediate or full phases of development across categories. 
Ratings for the implementation of goals and strategies tended to be moderate or 
intermediate for second cohort schools. Finally, low to intermediate levels of 
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implementation were apparent among third cohort schools in their initial year of 
operation. Across schools, the strongest levels of implementation tended to be for 
benchmarks targeting support and organization, whereas, lower levels of 
implementation were more apparent in the areas of curriculum, instruction, and 
evaluation. The school leadership team seemed to overestimate the extent of progress 
they had made in the implementation of the various benchmark goals and strategies, 
particularly as they related to curriculum and instruction. This tendency to overestimate 
progress signals a need to revise the benchmark document and become more 
comfortable with formative evaluation as a means for accurately assessing status and 
guiding improvement planning. 

In examining teacher reactions and experiences with charter schools, we found 
teacher responses remained primarily favorable. As reported in previous years, 
teachers were overwhelmingly positive about the school’s mission and program as 
evidenced by strong ratings in this category among all 12 schools. Compared to last 
year, there seemed to be some improvement in their experiences with professional 
development because this category received a strong ranking in all but three schools. 
Teachers tended to be less enthusiastic about available resources and support. 
Although there were differences by school in the type of resources that respondents 
endorsed as sufficient or adequate, having enough common planning time and faculty 
or staff more commonly emerged as areas in need of improvement. With regard to 
perceptions of support or assistance from educational agencies, teachers’ ratings and 
comments were less favorable. They consistently reported a lack of effective 
assistance provided by state and local educational agencies, and this finding may be 
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more of a reflection on these agencies rather than the charter schools. Regarding 
longitudinal trends, improvement in teacher responses were noted in several schools, 
and patterns related to cohorts were not detected. 

Parent satisfaction was another definite strength of all charter schools across all 
years. The areas for which parents expressed less satisfaction were often those related 
to the physical building, facilities, surroundings, transportation, and lack of additional 
extracurricular activities such as sports programs. These limitations are more difficult 
for schools to address because they require substantial resources. The levels of 
parental involvement were more varied across schools, with most considered to be 
strong or moderate in this category. There was a pattern of more moderate to low 
levels of involvement in third cohort schools, which would make sense for schools just 
launching their programs. The definition of what constitutes parental involvement 
remains elusive, especially when interpreting the qualitative data. If defined as support 
or communication, the levels of involvement would be considered stronger than if 
defined as active, regular participation in activities related to the education of their 
children. The difficulty associated with encouraging and maintaining high levels of 
parent satisfaction and involvement is not unique to charter schools, and arguably these 
charter schools have been more successful in this endeavor than many public schools 
serving similar types of students. 

These conclusions should be tempered with the recognition of the limitations 
associated with the evaluation data. Small sample sizes and low response rates 
diminish the validity of the findings. This limitation was particularly problematic in third 
cohort schools that had few respondents on the school climate and teacher surveys. 
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When making comparisons across years, the fact that there were different groups of 
respondents may have contributed to the changing trends across time. Finally, we 
based some results on the benchmark documents, which varied in quality and were 
prepared by the leadership team, not the evaluators. Every effort was made to base the 
findings on multiple data sources. 

Recommendations 

Due to a consistent pattern of findings across years, the recommendations are 
similar to those presented in previous reports. However, there are fewer 
recommendations this year because the areas identified as in need of improvement last 
year did not emerge as limitations globally affecting charter schools this year. Order or 
discipline issues and meeting the requirements of special needs students were not 
addressed this year because they emerged as limitations in a minority of the charter 
schools studied. This is not to say that these areas are no longer a problem in any of 
the charter schools. Rather, they did not emerge as limitations in the majority of 
schools. 

Based on the overall findings, the following recommendations apply to the 
charter schools as a group and not necessarily to individual schools. The 
recommendations specific to each school can be found in the individual school reports. 

Charter schools might adopt a wider array of instructional orientations or 
strategies shown to promote student achievement. Whereas there was improvement 
noted in some schools, there was little change in other schools or even an increase in 
more traditional instructional approaches. Many of the research-based practices and 
the schools’ own benchmarks call for the implementation of more student-centered 
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strategies that entail higher-order thinking and technology integration. Teacher led, 
direct instruction can be an effective orientation, but a better balance among a variety of 
effective pedagogical practices might be achieved. The low teacher-to-pupil ratios, 
perceived to be strengths of these charter schools, would seem to be an ideal 
environment for achieving this balance. The integration of technology into the 
classrooms to enrich learning and foster higher-ordering thinking may be a good starting 
point. 

Benchmark documents need to be modified to better align with objective 
indicators and available data. Continued improvement in developing and then attaining 
benchmarks was noted among earlier cohort schools. However, in other schools, 
especially third cohort schools, benchmark development might be improved. Benchmark 
documents are intended to be a dynamic tool used for school improvement. Planning, 
evaluation, and ultimately improvement can be facilitated by a sound, ambitious, and 
long-range benchmark document. Results and data from this report, independent 
classroom observations, and student outcome results exemplify some of the evidence 
that might be provided to document goal attainment. An honest, accurate appraisal of 
the extent to which benchmarks have been successfully implemented will enhance 
school improvement efforts. 

Efforts to continue developing supportive, collaborative relationship with the 
school district and external partners should continue. Across cohorts, the present 
findings have consistently indicated a lack of effective assistance from local and state 
educational agencies and, in some cases, external partners. The educational reform 
literature clearly emphasizes the importance to the sustainability of school change of 
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having close connections to the parent school district. Strategies include having more 
open channels of communication and holding meetings to clarify policies, procedures, 
and available resources. Furthermore, representatives of charter schools should 
continue to seek the support and assistance of various external partners. Publicity and 
notices in the form of newsletters, invitations to school functions, and community events 
might be used in conjunction with direct requests for assistance to promote more 
involvement from businesses, community groups, colleges, and parents. However, we 
also recognize that the schools’ potential progress in establishing and furthering such 
relationships is directly dependent on the degree of involvement desired and investment 
made by the respective external organizations 

Although improvement in the extent of parental involvement was apparent in 
some schools, in others it was at moderate or low levels. Therefore, another 
recommendation is to continue efforts to increase active parent involvement or 
participation. As noted in previous years, schools that had formal contracts of 
agreement with parents specifying the minimum number of hours of participation 
required (COSLA, Star, and Southern) tended to have more parental involvement. 
Furthermore, the definition and types of parental involvement might be reconsidered. In 
some school improvement plans, involvement translates into support for and 
understanding of the mission and educational program or maintaining regular contact. 
In others, it specifies more active participation in parent organizations, school activities, 
and the provision of academic support for students. Both are necessary and important, 
and both might be included on benchmark documents that identify more specific 
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strategies used to encourage involvement coupled with more objective evidence to 
indicate their effectiveness. 

The final recommendation is to prioritize, coordinate, and secure more resources. 
Because the specific type of resource needed varied by individual schools, resources 
are broadly defined. In some schools limited resources in the areas of technology, 
educational materials, and common planning time were noted. In others, the need for 
more faculty and staff was cited. Parents tended to suggest changes that required 
substantial resources such a new larger school building that would house a library, 
gymnasium, or cafeteria. The selection of targeted resources would be guided by the 
school improvement plan. Admittedly, this recommendation is the most difficult to 
achieve. Additional resources may be garnered by working toward achieving a 
collaborative, supportive relationship with the school district, state educational agency, 
and external partners. 
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School Observation Measure (SOM) Data Summary 

National Norms for Elementary Schools 



Number of School Observation Visits for N = 688 

Note: One school observation visit equals approximately 10 individual classroom visits. 



The extent to which each of the 
following was used or present in the 
school... 


Percent 

None 


Percent 

Rarely 


Percent 

Occasionally 


Percent 

Frequently 


Percent 

Extensively 


Instructional Orientation 


Direct instruction (lecture) 


3.0 


10.4 


23.7 


34.8 


28.1 


Team teaching 


49.8 


32.3 


11.9 


5.6 


0.4 


Cooperative/collaborative learning 


38.1 


41.8 


13.1 


5.2 


1.9 


Individual tutoring (teacher, peer, 
aide, adult volunteer) 


44.6 


31.8 


16.1 


5.2 


2.2 


Classroom Organization 


Ability groups 


36.3 


22.5 


15.7 


12.7 


12.7 


Multi-age grouping 


70.1 


15.7 


7.5 


4.1 


2.6 


Work centers (for individuals or 
groups) 


27.3 


27.7 


26.2 


15.4 


3.4 


Instructional Strategies 


Higher-level instructional feedback 
(written or verbal) to enhance student 
learning 


28.0 


36.2 


26.5 


7.1 


2.2 


Integration of subject areas 
(interdisciplinary /thematic units) 


58.3 


30.7 


7.6 


3.0 


0.4 


Project-based learning 


75.4 


15.8 


5.0 


3.8 


0.0 


Use of higher-level questioning 
strategies 


19.5 


32.7 


32.0 


12.4 


3.4 


Teacher acting as a coach/facilitator 


7.1 


20.2 


28.5 


34.5 


9.7 


Parent/community involvement in 
learning activities 


75.3 


18.4 


6.0 


0.4 


0.0 


Student Activities 


Independent seatwork (self-paced 
worksheets, individual assignments) 


4.1 


13.7 


39.6 


31.1 


11.5 


Experiential, hands-on learning 


30.5 


36.4 


24.9 


6.3 


1.9 


Systematic individual instruction 
(differential assignments geared to 
individual needs) 


64.9 


26.5 


5.6 


2.2 


0.7 


Sustained writing/composition (self- 
selected or teacher-generated topics) 


56.0 


32.8 


7.8 


2.2 


1.1 


Sustained reading 


43.3 


31.9 


18.5 


4.1 


2.2 


Independent inquiry/research on the 
part of students 


82.8 


13.1 


2.6 


1.5 


0.0 


Student discussion 


46.0 


24.5 


16.6 


9.1 


3.8 
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National Norms for Elementary Schools, CONTINUED 



Number of School Observation Visits for N = 688 

Note: One school observation visit equals approximately 10 individual classroom visits. 



The extent to which each of the 
following was used or present in the 
school... 


Percent 

None 


Percent 

Rarely 


Percent 

Occasionally 


Percent 

Frequently 


Percent 

Extensively 


Technology Use 


Computer for instructional delivery 
(e.g. CAI, drill & practice) 


35.2 


37.1 


20.6 


6.7 


0.4 


Technology as a learning tool or 
resource (e.g., Internet research, 
spreadsheet or database creation, 
multi-media, CD Rom, Laser disk) 


49.6 


29.9 


16.4 


3.4 


0.7 


Assessment 


Performance assessment strategies 


66.4 


16.4 


9.9 


5.7 


1.5 


Student self-assessment (portfolios, 
individual record books) 


72.9 


18.7 


5.7 


2.7 


0.0 



Summary Items 


High academically focused class time 


0.4 


1.5 


17.1 


38.3 


42.8 


High level of student 
attention/interest/engagement 


0.4 


3.0 


20.1 


48.3 


28.3 



Rubric for SOM Scoring 

(0) None: Strategy was never observed. 

(1) Rarely: Observed in only one or two classes. Receives isolated use and/or little time in classes. 

Clearly not a prevalent/emphasized component of teaching and learning across classes. 



(2) Observed in some classes. Receives minimal or modest time or emphasis in classes. 

Occasionally: 

Not a prevalent/emphasized component of teaching and learning across classes. 



(3) Observed in many but not all classes. Receives substantive time or emphasis in classes. 

Frequently: 

A prevalent component of teaching and learning across classes. 



(4) Observed in most or all classes. Receives substantive time and/or emphasis in classes. 

Extensively: 

A highly prevalent component of teaching and learning across classes. 
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School Observation Measure (SOM) Data Summary 

National Norms for Secondary Schools 



Number of School Observation Visits for N = 370 

Note: One school observation visit equals approximately 10 individual classroom visits. 



The extent to which each of the 
following was used or present in the 
school... 


Percent 

None 


Percent 

Rarely 


Percent 

Occasionally 


Percent 

Frequently 


Percent 

Extensivel 

y 


Instructional Orientation 


Direct instruction (lecture) 


1.6 


8.7 


19.0 


44.0 


26.6 


Team teaching 


64.1 


30.3 


5.4 


0.3 


0.0 


Cooperati ve/coll aborati ve le anting 


35.7 


36.5 


23.2 


4.4 


0.3 


Individual tutoring (teacher, peer, 
aide, adult volunteer) 


76.5 


17.8 


1.9 


3.0 


0.8 


Classroom Organization 


Ability groups 


40.1 


29.8 


13.3 


7.3 


9.5 


Multi-age grouping 


53.7 


10.6 


8.9 


17.1 


9.8 


Work centers (for individuals or 
groups) 


69.4 


23.0 


7.3 


0.3 


0.0 


Instructional Strategies 


Higher-level instructional feedback 
(written or verbal) to enhance student 
learning 


25.1 


24.9 


25.7 


18.1 


6.2 


Integration of subject areas 
(interdisciplinary /thematic units) 


66.1 


21.7 


8.9 


1.4 


1.9 


Project-based learning 


51.1 


35.1 


10.6 


2.4 


0.8 


Use of higher-level questioning 
strategies 


19.2 


29.0 


32.8 


17.1 


1.9 


Teacher acting as a coach/facilitator 


3.8 


16.3 


27.9 


28.5 


23.6 


Parent/community involvement in 
learning activities 


93.0 


6.0 


1.1 


0.0 


0.0 


Student Activities 


Independent seatwork (self-paced 
worksheets, individual assignments) 


1.1 


9.8 


26.0 


39.6 


23.6 


Experiential, hands-on learning 


44.0 


36.4 


14.7 


4.6 


0.3 


Systematic individual instruction 
(differential assignments geared to 
individual needs) 


76.1 


20.9 


2.7 


0.3 


0.0 


Sustained writing/composition (self- 
selected or teacher-generated topics) 


54.5 


33.2 


10.4 


1.6 


0.3 


Sustained reading 


52.5 


36.0 


10.7 


0.8 


0.0 


Independent inquiry/research on the 
part of students 


67.7 


24.5 


7.1 


0.8 


0.0 


Student discussion 


65.7 


17.2 


8.7 


4.6 


3.8 
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School Observation Measure (SOM) Data Summary, continued 
National Norms for Secondary Schools 



The extent to which each of the 
following was used or present in the 
school... 


Percent 

None 


Percent 

Rarely 


Percent 

Occasionally 


Percent 

Frequently 


Percent 

Extensively 


Technology Use 


Computer for instructional delivery 
(e.g. CAI, drill & practice) 


49.6 


40.1 


8.1 


2.2 


0.0 


Technology as a learning tool or 
resource (e.g., Internet research, 
spreadsheet or database creation, 
multi-media, CD Rom, Laser disk) 


45.5 


37.9 


11.9 


4.6 


0.0 


Assessment 


Performance assessment strategies 


59.2 


26.3 


11.8 


2.7 


0.0 


Student self-assessment (portfolios, 
individual record books) 


77.7 


17.1 


4.1 


1.1 


0.0 



Summary Items 


High academically focused class time 


0.0 


1.6 


23.4 


59.4 


15.5 


High level of student 
attention/interest/engagement 


0.0 


3.3 


42.0 


46.9 


7.9 



Rubric for SOM Scoring 

(0) None: Strategy was never observed. 

(1) Rarely: Observed in only one or two classes. Receives isolated use and/or little time in classes. 

Clearly not a prevalent/emphasized component of teaching and learning across classes. 



(2) Observed in some classes. Receives minimal or modest time or emphasis in classes. 

Occasionally: 

Not a prevalent/emphasized component of teaching and learning across classes. 



(3) Observed in many but not all classes. Receives substantive time or emphasis in classes. 

Frequently: 

A prevalent component of teaching and learning across classes. 



(4) Observed in most or all classes. Receives substantive time and/or emphasis in classes. 

Extensively: 

A highly prevalent component of teaching and learning across classes. 
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